Problem with my algorithm

[electroRF]

hi NorthGuy,
Thank you again my friend.

that sounds a good option.

ill put more thought on it to put it on paper and will raise it up in front of my team.

ill need before to offer quick solution for my current algorithm.

what do you think of this method to detect that i crossed to the4th quarter?

//assuming 16k buffer
Accumulator = R - BUFFER_START
ACCUMULATOR &= (1<<13 | 1<<12); //Mask with 8kB and 4KB
if (Accumulator == (1<<13 | 1<<12) )
Transfer_1st_half

 

[NorthGuy]

Based on the information you disclosed, I don't see any benefits of doing 1/4 buffer compare to 1/2 buffer. DMA seems to be fast enough for both. 1/2 buffer solution will produce less DMA initiations, which you seem to think are bad. What is the reason to go to 1/4 buffer?

 

[electroRF]

Hi NorthGuy,
I thank you again for your wise notes!

I mis-explained myself.

Right now, I'm triggering the DMA every half buffer, i.e. every 5KB.
Every time I reach the 2nd half, I trigger the DMA to transfer the 1st half.
Every time I reach the 1st half, I trigger the DMA to transfer the 2nd half.

I want to keep triggering the DMA to transfer half buffer.
But, now, I want to do one change,
(Buffer Size is 10KB)
When I reach address (Start_Of_Buffer + 7,500), I wanna transfer the 1st half (0-5KB),
When I reach address (Start_Of_Buffer + 2,500), I wanna transfer the 2nd half (5-10KB).

I thought of doing it that way:

int Counter = 0; //Global Counter
int SrcAddrArr[2] = {BUFFER_START_ADDRESS, BUFFER_MID_ADDRESS}
int SrcAddrIndex = 1;

/*** On Each Logging Function ***/

Counter += Len; //Increment the Counter by Len of each Message
if (Counter >=7168) //7168 = 7KB
{
    TriggerDma(SrcAddrArr[SrcAddrIndex], DestAddr); //DMA Transfers 5120Bytes from SrcAddrArr[SrcAddrIndex] to DestAddr
    Counter -= 5120;
    SrcAddrIndex ^= 1;
}

What do you think of it?

Thank you NorthGuy.

 

[NorthGuy]

I see. You want to delay DMA transfer to make sure all the writes into the small memory are complete. This, of course, will better chances of all the small memory being written by the time you do the transfer. It is hard to tell if this is going to completely eliminate the problem, but from this viewpoint it's definitely better than 1/2.

If you move it further, say to 7/8 from 3/4, it decreases the chances of non-complete writes even further, but increases the chances of DMA not being complete. You also can decrease it to 5/8. It's sort of a compromise.

Looking at how the whole thing is working, I would rate it somewhere between "bad design" and "complete disaster". The memory structures that are used were created for completely different use. The delays in writting small memory are far greater than you thought so when the system gets busy (when your service might be needed the most), the outcome is not guaranteed, but only probable. The writing performance (one of the design goals) is optimized for the use of circular buffer and might be improved considerably with better memory organization, decreasing the toll on the system.

In such situation, the best idea is to redesign from scratch using all the information you now have. It's not like you wrote 1oK lines of code and now need to re-write it all, although such things happen too Spending a day to re-design and re-write it would be well worth it.

 

[electroRF]

Hi NorthGuy,

I completely understand you, thank you very much.

I'm not sure though, you mean to design the system by creating a pool of buffers, each with a destination address.

However, the large buffer is 100 times bigger than the small 10KB buffer.

So, you mean that each buffer in the pool will have several destination addresses?

Also, How would you efficiently tell which one of the buffers in the pool is free?

 

[NorthGuy]

A very efficient way to manage buffers is to put them in a double-linked lists. A list of free buffers and a list of buffers pending transfer. But these are technical details and this is not a good place to start with.

The first question about the design is what it is supposed to accomplish. As I understan this is some sort of a log which is to be used to study system behaviour in case of a crash, or just if someone wants to stop the system and debug. As I understand this is a preemtive environment where any thread can be switched on or off at any time. The requests are coming from different threads at different times, which is largely unpredictable. They have to be ordered and timestamping.

I would start from this. You receive a request. Definitely, the first thing is that you want to timestamp it. It is possible that before you timestamp it, threads get switched, so your timestamp will only happen after the thread gets control back, possibly several us (even ms) later. So, the best we can do is approximate timestamping and therfore not very strict ordering. Correct?

Further. Once you get a request, how important is to push it through. Is it Ok to put it on hold? Say, you get it and few us later the system crashes. It's likely that the request you received is somehow important because it is the last request and good chances are it is somehow related to the crash. Is it important to preserve information about this request? If so, you probably cannot wait few ms until the writing thread gets its control back, because this may never happen because of the crash. What are your thoughts about this?