Pic, RS232 and Simulation

[Dakta]

Hi guys, long time no see. Sorry for the long post, just seeking a bit of theory and practical knowledge really from anyone used to working with serial comms.

I have a few questions regarding the approach I should take to tackling a problem. Basically I want to log data to my PC from a couple of sensors, and store basic configuration data (such as refresh times just as an example) on the PIC.

I have no trouble actually setting up the pic (16f88) and having read the datasheet I've got a few examples up and running without too much hassle.

What I thought i'd do is have a (I think we call it a rolling stack?) array of bytes, say 8 bytes wide, and every time a new byte is received by the uart module, it would raise an interrupt, push it onto the bottom of the stack (position 7) and shuffle all the variables up by one position, eventually popping the top value off the top.

Then the top five bytes would be checked for a header signalling the start of a command, say '255, 255, 0, 0, 128', if they are present then the remaining three bytes at the bottom of the stack would be used to construct a command, one value being to signal the command, the other two being used to provide data for that command.

So basically it's a mini command interpreter. The reason why I've chosen this is because I want two way communication and multiple sensors, I need a way for the pic and the pc to identify between different requests. The pc is going to want to both read and write eeprom randomly and the pic is going to want to send sensor data based on whats being requested.

I suppose my first question is, is this a correct design philosophy? (i.e having a command structure) - or am I missing an obvious technique for defining the meaning of received data?

Finally, there is a slight problem - I wrote an example to test this theory, I wrote the firmware in MikroC, and I built a vb.net application to generate commands and interpret and display received commands. I simulated this set up in proteus using a virtual serial port 'pair'. In essence it works - if I send a command from one, the other receives it. The problem is, every now and then incorrect data is received, well actually, the data simply isn't received at all. I've verified from sniffing the datastream that the simulated pic is receiving all the data, but sometimes the pic doesn't respond.

Currently about 90% of data is being sent/received without a problem.

The baud rate is currently set a 933, data transfer rate isn't terribly critical, and with a 4mhz clock this gives an error of 0.0%. I was wondering if the load on the pc whilst simulating my pic and my vb.net program could be causing timing inconsistencies. Is this a logical explanation? I'm currently waiting for a protoboard that will enable me to test the system in hardware but it'l be a while before it arrives.

Secondly, another consideration I have is that perhaps checking for commands in the interrupt routine is taking too long, and thus the pic is missing data. However creating delays inbetween the sending of bytes has not had an improvement on the overall accuracy of data. So I'm a bit dubious about this theory.

What I do know is - if I transmit the same command-sequence of bytes twice, the chance of the command being executed is (almost) doubled.

So my question really is, does this sound like a baud rate/timing problem?

I don't want to post my entire code here (I can if you really want though) because my program is quite long and possibly incomprehensible simply due to it's length, I'm just seeking opinions really.

 

[mynameisdan]

I would use a simple array call it "stack a", which you can expand and shrink at will. Use a struct to contain the data, and send the struct itself to the computer. I would then stuff the struct into another struct that contains a simple CRC of the data also a time stamp, which is also in a stack "stack b".

The computer should reply to all packets with the CRC. The PIC can remove packets with a valid CRC and resend packets which timeout.

 

[Dakta]

That's a pretty nifty idea, I'm going to look into that.

I've spent the last couple of days just playing with it, this might sound daft but introducing a millisecond delay between sending each byte has increased accuracy from about 60% to 90%. So I'm still wondering if it's delays caused by running a simulation on what I admit isn't the most powerful computer that might be doing it. By repeating commands twice, accuracy shoots up yet again, with one byte lost (not corrupted, simply never received) out of every 4000 sent approx.

I'm starting to wonder if a pic16 is a suitable controller for the job too. The reason I didn't provide code is because it's part of a fairly complex solution that does a lot of calculations when left to it's own devices (and uses eeprom contents as elements of these calculations) so setting the config via rs232 really wants to be a small part of the program. I try to modularise the code, every 'task' is a function, however the compiler isn't happy with re-entrancy should I call the same function from two places in code due to the stack, so I've got to dodge my way around this too. I might look into getting started with the pic18's, especially since they cost less. Just a bit daunted by the amount of config options

Anyway, i'm going to experiment with your suggestions, and I'm constructing a demo circuit so I can test it 'physically' so touch wood we'll get to the bottom of it all cheers.

 

[Nigel Goodwin]

That's a pretty nifty idea, I'm going to look into that.

I've spent the last couple of days just playing with it, this might sound daft but introducing a millisecond delay between sending each byte has increased accuracy from about 60% to 90%. So I'm still wondering if it's delays caused by running a simulation on what I admit isn't the most powerful computer that might be doing it. By repeating commands twice, accuracy shoots up yet again, with one byte lost (not corrupted, simply never received) out of every 4000 sent approx.

I'm starting to wonder if a pic16 is a suitable controller for the job too. The reason I didn't provide code is because it's part of a fairly complex solution that does a lot of calculations when left to it's own devices (and uses eeprom contents as elements of these calculations) so setting the config via rs232 really wants to be a small part of the program. I try to modularise the code, every 'task' is a function, however the compiler isn't happy with re-entrancy should I call the same function from two places in code due to the stack, so I've got to dodge my way around this too. I might look into getting started with the pic18's, especially since they cost less. Just a bit daunted by the amount of config options

As a 'halfway move', what about the enhanced 16F series, such as the 16F1827 which I've recently started using - much cheaper then the older 16F, and with enhanced capabilities.

 

[Dakta]

What a good idea, just had a quick look on the microchip site and yeah, that looks to be the better idea. Heck, I'm just scanning through the datasheet, is that a DAC I see?

Pin compatible too, don't need a DAC for this project but I'm mighty impressed, seems to pack a lot of punch!

 

[Nigel Goodwin]

What a good idea, just had a quick look on the microchip site and yeah, that looks to be the better idea. Heck, I'm just scanning through the datasheet, is that a DAC I see?

Pin compatible too, don't need a DAC for this project but I'm mighty impressed, seems to pack a lot of punch!

It's full of nice goodies, and cheap too - in ones only £0.97 where the 16F648a is £2.20.

The DAC is only low resolution though.

What's REALLY nice is linear access to the GPR's, I'm using most of it as a 240 byte RS232 receive buffer.

 

[Dakta]

Pin compatible too - (though I'll have to watch out for the alternate pin registers) Thanks, don't seem to be available on ebay (my usual haunt for chips I'll be honest), Microschip have some shocking shipping prices...but....future electronics, (who are the only stockist I can find for a MPXH6400AC6U 400kpa pressure sensor I'm using for the same project (who also charge a fair whack for shipping) have them for 86p each.

Which means if I combine the order, and buy two pic's (one spare) I've actually saved money over the same project with a 16f88....funny how it works isn't it!

I wouldn't have thought of that, i just used the first chip that fell within the specs of what I wanted to do. It's been a lot of hassle working with the F88, for example, I'd have two functions, one for interpreting one command received via RS232 comms, and the other function for interpreting another, if both commands required a byte to be sent back in response, it wouldn't let me use the built in library in each function, so i'd either have to write my own, accessing the registers directly, or I'd have to set flags and deal with it all in one routine in the main loop, which would mean my functions for dealing with received commands would only do half a job, leaving my main code to do the tidying up after it! madness!

As you can tell, that makes what's already a complicated project almost impossible to navigate! (anyone fancy a session of information overload? )

Thanks bud, what a great suggestion. Didn't know that little beastie existed.

edit: not to be nosy, but how are you structuring your 240 byte buffer? do you use a 'pointer' to the address of the variable you want, or do you push all the values up everytime you receive a new byte? basically 'how does it work? (in concept)'

I assume the advantages are that you can keep your interrupt routine code small, whilst receiving a fair bit of data? (rather than having to munch through it before the next byte comes).

 

[Nigel Goodwin]

Pin compatible too - (though I'll have to watch out for the alternate pin registers) Thanks, don't seem to be available on ebay (my usual haunt for chips I'll be honest), Microschip have some shocking shipping prices...but....future electronics, (who are the only stockist I can find for a MPXH6400AC6U 400kpa pressure sensor I'm using for the same project (who also charge a fair whack for shipping) have them for 86p each.

Which means if I combine the order, and buy two pic's (one spare) I've actually saved money over the same project with a 16f88....funny how it works isn't it!

I wouldn't have thought of that, i just used the first chip that fell within the specs of what I wanted to do. It's been a lot of hassle working with the F88, for example, I'd have two functions, one for interpreting one command received via RS232 comms, and the other function for interpreting another, if both commands required a byte to be sent back in response, it wouldn't let me use the built in library in each function, so i'd either have to write my own, accessing the registers directly, or I'd have to set flags and deal with it all in one routine in the main loop, which would mean my functions for dealing with received commands would only do half a job, leaving my main code to do the tidying up after it! madness!

As you can tell, that makes what's already a complicated project almost impossible to navigate! (anyone fancy a session of information overload? )

Thanks bud, what a great suggestion. Didn't know that little beastie existed.

edit: not to be nosy, but how are you structuring your 240 byte buffer? do you use a 'pointer' to the address of the variable you want, or do you push all the values up everytime you receive a new byte? basically 'how does it work? (in concept)'

I assume the advantages are that you can keep your interrupt routine code small, whilst receiving a fair bit of data? (rather than having to munch through it before the next byte comes).

It's for a GSM modem product, which is coming on VERY nicely (I've currently got the GSM chip we're using talking to the PIC, and doing what it's supposed to)

Interrupts are used to keep a real time clock running.

I simply write the entire incoming text message to the buffer, using polling, no interrupts for this routine - and if no data has arrived for 255mS odd, then it's finished coming. I then process the buffer to search and extract the parts I want, essentially I'm checking for two text strings, one to set the number it sends reports to, and the other to set the location it's going to be installed at. The same routines also check for signal strength reported by the modem chip, and that's added to the outgoing texts as well, as is a count of how many texts it's sent in total. Next I've got to add a battery voltage test as well, and send that in the text - it's nice that the 16F1827 has internal volage references, so I can measure Vdd with just a couple of resistors.

 

[Dakta]

Sounds interesting but a little bit beyond me at the moment

Unfortunately it turns out I don't have the software for simulating the 16f1827, so i'm going to have to wait until I can complete my physical circuit before porting my code over, it clearly is the superior chip - shame about the DAC though I got a bit excited (not come across one on a pic before) but 32 levels ruined the fun a bit

Oh well, not necessary for this project, but would have been fun as I can think of good uses for being able to monitor signal inputs 0-5v and generate slightly modified outputs reflecting the input, for example a circuit for reducing the fuel quantity feedback on diesel engines, reducing the signal slightly at high loads to give a bit extra performance (most people just use a resistor on the fuel pump, works well but problem is you get a hunting idle as you increase idling fuel quantity too). That could be a good project for a bit of extra refinement maybe. Anyway, I digress.

Question - in your gsm application, you didn't mention if you did any error checking. Based on responses so far, and from what I've read doing a bit of background reading on other RS232 pic projects, I get the impression most people do apply a routine for error checking etc (checksum and so on). Does your solution employ this? What I suppose I'm asking is, is it unreasonable to expect data to be received properly when your pic is correctly timed (0% error apparently) with the baud rate you are using?

So I suppose in other words, is it the nature of the beast to have dropped packets every so often?


And finally, good luck with your project. I tried to give you a positive rep or 'thanks' but it wouldn't let me, need to spread it around apparently. lol.

 

[Nigel Goodwin]

Question - in your gsm application, you didn't mention if you did any error checking. Based on responses so far, and from what I've read doing a bit of background reading on other RS232 pic projects, I get the impression most people do apply a routine for error checking etc (checksum and so on). Does your solution employ this? What I suppose I'm asking is, is it unreasonable to expect data to be received properly when your pic is correctly timed (0% error apparently) with the baud rate you are using?

So I suppose in other words, is it the nature of the beast to have dropped packets every so often?

Some kind of error checking is normal in RF links, or modem links down phone wires (but that's internal to the modem anyway, and not something you have to write).

And finally, good luck with your project. I tried to give you a positive rep or 'thanks' but it wouldn't let me, need to spread it around apparently. lol.

There's no error checking, as it's only travelling along a few cm's of track (or wire) there's no need, and the modem chip doesn't support error checking on it's input anyway.

 

[Dakta]

I've just had a nasty thought actually, I use a fairly old programmer and I do not think it will support the 16F1827, or other newer pics.

Is there a programmer that can? (I suppose 'tight budget' being the operatives here). There is apparently a software bodge you can do to get my programmer to work, but I'll assume worst case that it doesn't work (as results seem mixed).