Data analysis from a sensor

[EngIntoHW]

Hi
I have a question please. Apologies that it's go to the analysis part of it rather then electronic part.
I have a bunch of livestock with an ear sensor on each, that sends me data about how much heat is going out of the ear.

I wanna take this data from each livestock and smooth it.

My question is, how to know which smoothing algorithm I should use to get an authentic reliable data?

Thank you.

 

[danadak]

Various algorithms produce various results. You have to pay attention
to detail for each algorithm. More recent applications showing up
using wavelets to overcome some of the loss of characteristics of
signals.

When you do simple smoothing, like sample set moving or running
average, you can then do error fit which is useful in prediction. Least
Squares and Power Curve fit quite common.

Not a mathematician so time for one to step into this conversation.


Regards, Dana.

 

[rjenkinsgb]

A simple approach I use a lot for such as noise reduction on relatively slow changing analog signals is just to average a number of samples; using a power of two makes it simple to program.

For each data source, you need two variables; an average accumulator and an output store, both big enough to hold the maximum input value * number of samples, without overflow.

eg. using 16x samples:

Initialise the output store with either the first data value, or a typical mid range value.
Multiply by 16 and store in the average accumulator.

In the main program: For each new sample, add that to the accumulator, then subtract the output store from the accumulator.

Divide the new accumulator value by 16 (eg. shift right four bits, save the result in the output store; that's also your smoothed value.

You could just as well use eg. four samples & shift two bits, or 64 and shift 6 bits; it's down to how much filtering you need and how much delay you can tolerate in any sudden changes of the input data becoming significant at the output of the averaging process.

 

[danadak]

Some application info on averaging, attached.


Regards, Dana.

 

[Pommie]

An alternative is a running average. Using 16 values again, subtract 1/16th from the accumulated value (x-=(x>>4)) and add the new value (or multiply by 15/16ths). The average is the accumulated value divided by 16.
This has the advantage that it's quicker as it only needs 1 new sample.

Actually, I now believe that is what rjenkinggb is describing above only with a slightly different algorithm.

Mike.

 

[Les Jones]

I have used a variation of that method. I have a ring buffer of the last 16 entries. The new reading is written into the next location in the buffer. (Overwriting the value 16 entries before it.) After it reaches the last location in the buffer the 4 bit pointer overflows back to zero (Pointing back to the start of the buffer.) After each new entry into the buffer the 16 locations are added together. This value is divided by 16 by doing 4 right shifts. This will only work if you don't need the fractional part of the result. (As you discard the last 4 bits of the sum of all 16 locations.

Les.

 

[Diver300]

I often use the algorithm that rjenkinggb and Pommie describe, in both microcontrollers and spreadsheets.

In microcontrollers it has the advantage of only needing to store one value. The behaviour can be a bit odd if the time difference between samples varies.

It one type of Infinite Impulse Response filter. https://en.wikipedia.org/wiki/Infinite_impulse_response