1. Welcome to our site! Electro Tech is an online community (with over 100,000 members) who enjoy talking about and building electronic circuits, projects and gadgets. To participate you need to register. Registration is free. Click here to register now.

Decoding and graphing BTc encoded data

Discussion in 'General Electronics Chat' started by vsmGuy, Aug 19, 2009.

  1. vsmGuy

    vsmGuy New Member

    Joined:
    Jun 25, 2006
    Messages:
    54
    Likes:
    1
    I am amazed by RB's work, especially the BTc encoder!

    I have been meaning to decode and graph BTc encoded data (arrays generated by his encoder) on a COMPUTER.

    I have modelled a RC filter using PERL as this allows quick prototyping and, unfortunately, the kind of graphs I get is not exact (maybe similar to my un trained eye, except at extreme ends).

    Also unlike the BTc encoder drawn waveform, the waveform I see from my data starts to oscillate between the power rail and ground after some time.

    Here is how I model the RC filter :

    Code (text):
    V(t) = V_FINAL + (V_INITIAL - V_FINAL) * (e ** (-t / RC ));
    where,
    VFINAL will be 0V if bit to be sent is low. It will be VCC if bit to be sent is high
     
    The value of RC is the value suggested by RB's application :

    R = 2966E
    C = 0.22uF

    Hence, in my code I have

    Code (text):
    my $RC = ( 2966 ) * ( 0.22 / ( 10 ** 6 ));
    This is exactly what the subroutine "V" in my code returns:

    Code (text):
    my $samplingFreq = 11000; # In Hz
    my $samplingPeriod = ( 1 / $samplingFreq ); # In seconds

    my $e = 2.7186362391351841722168029758227;

    my $RC = ( 2966 ) * ( 0.22 / ( 10 ** 6 ));

    my $numberOfSamples = 17464;

    # Returns V(t) = V_FINAL + (V_INITIAL - V_FINAL) * (e ** (-t / RC ));
    # VFINAL will be 0V if bit to be sent is low. It will be VCC if bit to be sent is high

    my $OUTPUT_AMPLITUDE = 5;

    sub V
    {
        my ( $t, $BIT_TO_BE_SENT,  $V_INITIAL ) = @_;
       
        my $V_FINAL = $OUTPUT_AMPLITUDE * $BIT_TO_BE_SENT;
       
        $t *= -1;
       
        return ( $V_FINAL + (( $V_INITIAL - $V_FINAL ) * ( $e ** ( $t / $RC )))); # Models rise as well as decay
    }
     
    I put RB's encoded data in an array called 'sound_data' and I have a loop that iterates through it a byte (element) at a time (outer for loop), extracting a bit for every time period (inverse of the sampling frequency) from the MSB to the LSB (inner for loop) before progressing to the next bit.

    This loop writes out the time and amplitude in comma separated format that I later load into Excel and graph the data, and try and match it with the graph shown by RB's Encoder application.

    Code (text):
    my $V = 0;

    for( my $dx = 0; $dx < @sound_data; ++$dx )
    {
        my $t = (8 * $dx) * $samplingPeriod;

        #for(my $bit = 1; $bit <= ( 2 ** 7 ); $bit <<= 1)
        for(my $bit = ( 2 ** 7 ); $bit > 0 ; $bit >>= 1)
        {
            $V = V( $t,  (( $bit & $sound_data[$dx] ) ? 1 : 0 ), $V);
       
            printf "%f, %f\n", $t, $V;
           
            $t += $samplingPeriod;
        }
    }
    My final target is to create WAV file from the encoded data (arrays of data).

    This should be easy to do once I have a 1:1 match of my generated graph with RB's graph. .. the place where I need some help right now :confused:

    Whole package up for grabs at :

    https://sites.google.com/site/vimalsshankar/btc

    I also attached images of "What it should be"(RB's BTC graph) and "What It is" (What Excel shows my data as)

    As you can see, my data shows that my filter starts osciallting widly just after a few samples and does not follow RB's BTC waveform as shown as all! :eek:

    Attached Files:

    Last edited: Aug 19, 2009
  2. Mr RB

    Mr RB New Member

    Joined:
    Jul 22, 2008
    Messages:
    4,717
    Likes:
    187
    Location:
    Out there
    Hi vsmGuy, and thanks for the compliments with my BTc Encoder. :)

    Decoding BTc 1bit sound back to 8bit wave is not easy at all. I've thought about it for a lot of years but there is really no easy way, that's why i have never included BTc->Wave decoding in any version of the Encoder. I am very interested in what you are doing and if it can lead to a successful decoding system.

    The problem is that the BTc encoding of the wave->1bit data encoding is very lossy. For every sound sample the encoder is generating both a 0 and a 1 bit, (based on the RC curve) then choosing the "predicted" bit that is closest to the original wave. The encoding algoithms are shown on my
    Roman Black's BTc 1bit Sound Encoding Algorithms
    (BTc Algorithms page)

    So each BTc bit will have an error, that may be + or - to the original sample, by quite a large amount. Then the next bit will have another error that will tend to correct the previous error(s), because every new bit chooses the best option (+ or -) to get back to (closest to) the original waveform.

    This randomness of the error is extremely hard to reverse engineer being that the + or - for that bit is unknown, as were the bits before it that were also unknown. Also the predictive nature of the algorithm means there is a time issue as the BTc bit is predicted in advance of the new wave bit.

    Sorry I don't know PERL so I cant help you much there, but one thing seems an issue is that my algorithm uses the middle part of the 5v (ie the orig wave is reduced to half amplitude and then biased at 2.5v centre, so it goes from 1.25v to 3.75v only). Your first diagram seems to have a 0v-5v swing which would not be possible given that RC time constant and that bitrate. If you change that it should get you back close to the BTc waveshape. Again, check my encoding algorithms on the link above, reproducing the BTc waveform from the BTc data should not be that hard.

    I really don't think you can reverse encode the BTc data back to a wave based on a per-sample system for the reasons I stated above. The only way I think it CAN work is to do it the same way your ear does, by a low pass averaging of the total waveform. By averaging each sample with the samples before AND after (remember it's predictive) you should be able to remove most of the PCM noise and get a re-coded (not decoded) wave that is hopefully something like the original wave.

    Hope that helps.
  3. vsmGuy

    vsmGuy New Member

    Joined:
    Jun 25, 2006
    Messages:
    54
    Likes:
    1
    Thanks a lot Roman for the detailed answer!

    Well what I am trying to do is simple really - it should mimic the waveform that your encoder application plays when one clicks on "Play BTC Sound".

    I am not trying to do anything else - I need to hear what the sample might sound like out of the uC output pins!

    What I want to do is to create offset pointers into sound. For example, say I record :

    "Counting up. Now going down"

    As you can see, we have a lot of data that is common :

    "Count" "ing"

    "go" "ing"

    So if I can save a pointer to ofsset and lenmgth into "ing" of "Count" "ing", I just need to store "go" and I get "go" "ing" for free!

    Something like that :)

    I noticed you also have a pointer creation facility in your tool - is this the same?

    To summrize - my tool will allow you and anyone else to play the NTc encoded binary output WITHOUT actual hardware :)

    Is there something available to do that right now?
  4. Mr RB

    Mr RB New Member

    Joined:
    Jul 22, 2008
    Messages:
    4,717
    Likes:
    187
    Location:
    Out there
    Hi, re the pointers, yes the new version of BTc Encoder v3.0 handles sound pointers for you, this was a feature people had asked for that allows you to join sounds into one "library" or just to play separate parts of sounds as you said above. It really makes the system useful for makng a PIC talk etc as it can speak any word from within the sound library.

    As for playing the BTc encoded 1bit sound, my software already does that (during/after encoding), there's a button for it.

    If you want to play other 1bit BTc data as a wave file, you can simulate the PIC hardware playback of the 1bit stream. Sorry there is no tool available that currently does playback simulation (apart from actual playback in real hardware).

    If you want to code up a simulator, you can reverse the encoding algorithm. From my algorithm page; here is the procedure for encoding 1bit sound at BTc8;

    Code (text):

    variables;
        NewSample = the new audio sample, scaled to 1/4-3/4 Vcc range
        LastVoltage = the cap voltage (audio output) from the last bit
        NewVoltage = the cap voltage (audio output) after we encode this new bit
        NewBit = the newly encoded BTc bit

    encoding;
    1. generate a 1bit;
        distance = (Vcc - LastVoltage) / 8
        Voltage1 = LastVoltage + distance
    2. generate a 0bit;
        distance = (LastVoltage - Gnd) / 8
        Voltage0 = LastVoltage - distance
    3. Find which bit is closest to the desired audio sample;
        if Voltage1 is closest to NewSample then; NewBit =1, and NewVoltage = Voltage1
        if Voltage0 is closest to NewSample then; NewBit =0, and NewVoltage = Voltage0
    4. Save the cap voltage and repeat for the next audio sample;
        LastVoltage = NewVoltage
        Repeat
     
    So to simulate playback from a PIC to a RC filter, you can basically reverse the encoding algoithm. This will simulate the analog voltage on the cap in the RC filter of the playback hardware. The simulation procedure would look like this;

    Code (text):

    variables;
        TheBit = the BTc bit (from your data file)
        LastVoltage = the cap voltage (audio output) from the last bit
        NewVoltage = the cap voltage (audio output) after we simulate this new bit (this is the 0-255 "wave file" output)
       
    1. Start at middle of waveform;
        LastVoltage = 128
        NewVoltage = 128

    2. Get the next bit;
        If bit==1; NewVoltage = (LastVoltage + ((256 - LastVoltage)/8) )
        If bit==0; NewVoltage = (LastVoltage - (LastVoltage/8) )

    3. Now NewVoltage is the "wave" output sample, so save it

    4. Keep a copy of the cap voltage
        LastVoltage = NewVoltage

    5. Go back to 2, until all bits are simulated.
     
    You should be able to make that a PERL script without too much hassle. The simulation does assume you know what algorithm as used in encoding (in this case was alg1) and assumes you know what the BTc fineness value was (in this case it was BTc8).

    Because of the binary nature of the time constant it's extremely easy to simulate the RC filter resonse, in that for every bit (every sample) the voltage on the cap moves 1/8th of the way towards MAX (for a 1 bit) and 1/8th of the way toward ZERO (for a 0 bit). It's 1/8th because it was encoded at BTc8 obviously.

    The simulation code above will give you an unsigned 8bit "wave" file with each sample from 0-255, however the waveform will slightly reduced and be closer to the centre of the range since it was originally encoded from a 1/2 amplitude waveform.

    Now that simulated waveform will NOT be a perfect reproduction of the original wave due to the lossy nature of 1bit encoding. It will contain quite a lot of PCM noise "aliasing" etc that normally in hardware is reduced by low-pass filtering in the amp and speaker. If you want to play this as a wave file you can low-pass filter this farily easily to get an IDEA of what it would sound like, one technique I used was just to average every wave sample with the sample before it, this smoothes much of the PCM noise and gives an approximation of how the BTc 1bit playback will sound in actual hardware after filtering.
    • Like Like x 1
  5. vsmGuy

    vsmGuy New Member

    Joined:
    Jun 25, 2006
    Messages:
    54
    Likes:
    1
    So you mean so say, that if I have a sample that sounds :

    "Counting up. Now going down"

    Then I can access "Count", "ting", "up", "go", "down" from my hardware as pointers into a look up table? :confused:

    I did click around the set offset options in your application but found it hard to understand.

    Had I exported the data to C/ASM after setting the pointers, would have got some different kind of output?

    Could you help me out a little bit by giving on overview of this option :)

    Yes, but I am trying to figure out how to play data from middle of the array and I have not figure out the pointer option in your application yet :eek:

    This is exactly what I am trying to do :D

    Thanks a lot Mr RB!

    If I still don't get the thing to work with all the information, maybe I should dig a well and die inside it :(

    Thanks again for sharing this wondeful stuff with us!
  6. Mr RB

    Mr RB New Member

    Joined:
    Jul 22, 2008
    Messages:
    4,717
    Likes:
    187
    Location:
    Out there