Continue to Site

Welcome to our site!

Electro Tech is an online community (with over 170,000 members) who enjoy talking about and building electronic circuits, projects and gadgets. To participate you need to register. Registration is free. Click here to register now.

  • Welcome to our site! Electro Tech is an online community (with over 170,000 members) who enjoy talking about and building electronic circuits, projects and gadgets. To participate you need to register. Registration is free. Click here to register now.

Writing Log messages into Memory

Status
Not open for further replies.

electroRF

Member
Hi guys,

I'd love to get your advises on the following challenge.

The developers would like to print log messages into memory.

I'd need to implement a log_print function, which takes as arguments: UINT16 Header_ID, UINT16* pData, UINT16 len.

This function would write the header + Data + time-stamp into memory.

Later, I'll need to implement a parser, which will take this data and will convert it into text, using a known database.

Several challenges exist (perhaps you come up with additionals?)

1. how to overcome a situation where while message A being sent, an interrupt occurs, and the ISR will print a message B?
in the memory, it will look like that:
Start of Msg A ---> Start of Msg B ---> End of Msg B ---> End of Msg A

How could the parser know to differentiate between Message B and Message A?
I'd like to try to use as little overhead bytes as possible, to spare Cycles and Memory Size. (Due to Real-Time, I can't disable
interrupt while sending a message)

2. When should I read the time-stamp? how to overcome a situation where while reading the timestamp, I get an interrupt?

3. if you're aware of additional challenges that need to be taken care of, please share :)

Thank you :)
 
1. You reserve the memory first (in an atomic operation). Then you work on writing it. If interrupt happens during the write, it will reserve a chunk of memory past the memory you already reserved:

C:
length = calculate_length(msg);
 
if (!enough_space(length)) {
  whatever();
}
 
ptr += length; // must be atomic
 
write_message(msg, ptr);

2. this is a problem for the routine that gives out timestamps, not yours
 
Hi North Guy,
Thanks a lot!

1. You reserve the memory first (in an atomic operation). Then you work on writing it. If interrupt happens during the write, it will reserve a chunk of memory past the memory you already reserved:

It's a great idea which i haven't thought of.

Why does
ptr += length;
need to be an atomic operation?

I should have mentioned, the writing to memory is cyclic, so I'd have to take it into account when reserving the memory 'length'.
 
Why does need to be an atomic operation?

Let's imagine it's implented as something like this

Code:
mov length, R1
mov ptr, R2
add R2, R1
mov R2, ptr

If an interrupt happens after line #2 when ptr is already read and the program from the interrupt tries to access it, it will get the same value of ptr and will allocate its buffer in there. When interrupt returns, you will overrun whatever this program wrote into memory.

Therefore, space reservation must be uninterruptable.
 
Hi North Guy,
Thank you again :)

I understand now why you must work in disabled interrupt mode when reserving 'length' space.

The problem is that I'm not allowed to do so, because the prints will take place many times during the program lifetime, and if I'd disable interrupts every time a print is sent, i'd significantly decrease the performance of the program.

I thought of adding one extra UINT16 word, that will indicate on a message start, but then, what happens in the case of:
Start of Msg A ---> Start of Msg B ---> End of Msg B ---> End of Msg A
 
You do not need to disable interrupts. You just need to use assembler instructions which do the reservation at once, so that an interrupt cannot stick into the middle. Look at the instructions available on your processor and figure out how to do an atomic increment.
 
Hi NorthGuy,

thank you very much!

I'll look into my Processor's Instruction manual.

What do you think of this solution, in order to handle the case of : Start of Msg A ---> Start of Msg B ---> End of Msg B ---> End of Msg A

Each message will be sent in this format:

1. Known_Header + Msg_Len + Data#1 + ... + Data #n
or

2. Known_Header+ Data#1 + ... + Data #n + Known_Trailer

That way, even if Message B is sent in the middle of Message A, the parser could distinguish between message A and message B.

The problem with that is, that the Known_Header (and Known_Trailer for 2nd method) must not be sent in any print as Data, otherwise it'd make the parser think that a new message started.

What do you think of that solution?
 
It doesn't really matter if you need to store a byte or any other length. You'll get the same problem. Like this:

The main program starts writting Msg_Len, so it wrote it in and then wants to increment the pointer. At this time, interrupt happens. The program from the interrupts writes its own Known_Header in place of Msg_Len etc. When it returns, the main program returns and overwrites all this with its own message.

You cannot bypass the need for an atomic operation.You can make your atomic operation smaller (e.g. flipping a bit), but you cannot completely eliminate it.
 
Hi North Guy,
I understand, thanks a lot!

Would 'reserving a length' prior writing the message, would be the only atomic operation needed during the entire message printing, is that correct?
 
Would 'reserving a length' prior writing the message, would be the only atomic operation needed during the entire message printing, is that correct?

Yes.

With circular buffer, you will need to make sure that during the interrupt it won't go full circle and won't overwrite the reserved memory "from behind".
 
Can I ask how the messages are received that a second message can interrupt a first? An obvious solution, in my mind, would be to only write to memory when the end of a message is received or did I miss something?

Mike.
 
Hi,
Thank you very much NorthGuy!

So at the beginning of Log_print function, I'd do:
C:
DISABLE INTERRUPTS; //C Macro for disabling Interrupts

     //Start of Disabled Interrupt Section
     pThisMessageFirstAddress = pGlobalNextAvailableAddr; 

     pGlobalNextAvailableAddr += length;   

    if (pGlobalNextAvailableAddr > pGlobalEndOfBuffer)
             pGlobalNextAvailableAddr -= BUFFER_SIZE;
    //End of Disabled Interrupt Section

ENABLE INTERRUPTS; //C Macro for enabling Interrupts

I'm not sure yet i can do the above 'Disabled Interrupt Section' in one atomic operation.


Hi Mike,
Thank you friend for the support.

Your suggestion is to set an 'BusyPrintingFlag' to one when starting to print a message, and clear it when finishing?

Regarding your question, say that Log_print starts writing Message A into memory.
Before finishing writing Message A, an interrupt occurs which also calls Log_print and starts writing Message B.
 
Last edited:
... say that Log_print starts writing Message A into memory.
Before finishing writing Message A, an interrupt occurs which also calls Log_print and starts writing Message B.

That is not a very good idea at all.
But, If you really need to allow one log_print to interrupt an unfinished log_print, maybe you should implement some kind of double buffering. When you start writing to one buffer, you switch the possible next message to go to another buffer. That would allow one level of interruption and keep every message intact.
 
Hi T,
Thank you.

Well, you can get several interrupts.
That way, 2 buffers are not enough.

What kind of buffer you have? Maybe each "interrupt" that is writing log allocates just enough memory from the buffer.. moving the pointer for "next starting message" to point to the end of that memory. Then the interrupt can fill the buffer without any other thread corrupting it.

For example something like this:

C:
interrupt() {
    uint16_t index; /* Holds the assigned buffer pointer */
    uint16_t message_length;
 
    message_length = /* Calculate message length */
 
    /* This next function gives a copy of the current "end" location of the buffer memory
      and then moves the internal "end" location forward by 'message_length' */
    /* If some other interrupt asks to write to the buffer, it will get an index
      pointing to the end of this message.. even if the message is not yet completely written */
    index = buffer_begin_write(message_length);
 
    /* Write to the buffer using the 'index' variable.
      The write function updates the 'index' */
    buffer_write(&index, "data");
}
 
Last edited:
Hi T,

Your suggestion of allocating enough memory in advance, is exactly what NorthGuy offerred, which is probably what I'd use.
e.g.
C:
DISABLE INTERRUPTS; //C Macro for disabling Interrupts

     //Start of Disabled Interrupt Section
     pThisMessageFirstAddress = pGlobalNextAvailableAddr;

     pGlobalNextAvailableAddr += length;

    if (pGlobalNextAvailableAddr > pGlobalEndOfBuffer)
             pGlobalNextAvailableAddr -= BUFFER_SIZE;
    //End of Disabled Interrupt Section

ENABLE INTERRUPTS; //C Macro for enabling Interrupts

Another solution to avoid Message A and B overriding each other would be to maintain a 'busy' flag.

What do you think?
C:
DISABLE INTERRUPTS;  
if (PrintingBusy == 0)  
{      
     PrintingBusy = 1;                      
     ENABLE INTERRUPTS;           
     Call printing function;      
     PrintingBusy  = 0;   
}  
else ENABLE INTERRUPTS;
 
Hi T,

Your suggestion of allocating enough memory in advance, is exactly what NorthGuy offerred, which is probably what I'd use.
e.g.

Haha.. sorry about that. The one sentence by NorthGuy got buried in all the talk about atomic operations. I need to read thread more carefully :)

I think the suggested solution is a good one. Easy to "add" to existing buffer.. just add couple of new functions. Doesn't need any extra memory etc.
 
Last edited:
Actually, I was a little bit wrong about atomicity.

In the result of your atomic operation, you should both increment and obtain the ponter to the buffer.

So this entire operation needs to be atomic

C:
pThisMessageFirstAddress = pGlobalNextAvailableAddr;
pGlobalNextAvailableAddr += length;

You're talking about interrupts. Is this the only problem? How about other threads?

Usually OS provides some sort of "critical section enter/leave" functions that let you forbid interruptions during the critical section. Does your OS have this?
 
Hi NorthGuy,
First of all, I'd like to thank you very much for your great advises! :)

Yes, you're right on the atomicity of both operations - Obtain the pointer to the Buffer, then Increment the Pointer to the Buffer by 'length'.
I'd add that you'd also need to include inside the atomicity a 3rd operation - 'taking care of the circularity of the Pointer to the Buffer'.

That's exactly how I interpreted your method:

C:
DISABLE INTERRUPTS; //C Macro for disabling Interrupts

//Start of Disabled Interrupt Section
     pThisMessageFirstAddress = pGlobalNextAvailableAddr;

     pGlobalNextAvailableAddr += length;

    if (pGlobalNextAvailableAddr > pGlobalEndOfBuffer)
             pGlobalNextAvailableAddr -= BUFFER_SIZE;
//End of Disabled Interrupt Section

ENABLE INTERRUPTS; //C Macro for enabling Interrupts


What you're actually saying regarding having other threads interrupting the Mechanism, you're right.

You're actually saying that it's not enough the prevent interrupts from happenning, but it's also needed to prevent other tasks from taking over, right?
 
Couple of points:
- I believe most (at least some) microcontrollers automatically disable interrupts when entering ISR.
- Usually multitasking is disabled when interrupts are disabled.
- If you are really going to have nested interrupts, and on top of that somehow another thread can take over while in ISR.. you are heading for trouble. Can't imagine how can system like that behave well, or work at all in the long run.

Why can't you disable interrupts while executing isr? I understand that missing interrupts is bad, but usually at least one interrupt can be queued. Having three interrupts firing almost same time is always trouble.. executing them nested is more trouble. You need a better design for your software. Design the interrupts to be as short as possible (do you really need to write logs in the isr?).
 
Last edited:
Status
Not open for further replies.

Latest threads

New Articles From Microcontroller Tips

Back
Top