"Creating variable" on runtime? - message decoding
Hi guys,
here is my problem description:
I receive messages over a bus. Suppose there is a function to read the messages storing them into a struct defined as follows:
typedef struct
{
ULONG timestamp;
BYTE ID;
BYTE data_length;
BYTE data[8];
} MSG_FRAME;
Depending on the message ID different messages represent different values for one project.
For example msg with ID 10 can include in the 8 bytes something like:
width: 6 bits
height: 6 bits
actpos: 12 bits
maxpos: 12 bits
minpos: 12 bits
range: 16 bits
total: 64 bits = 8 bytes
Printing the message is no big deal. But here comes the tricky part. I want to print out the specific information hidden in the 8 bytes. I can define the structures for every msg ID and compile the program with this "special" header file, but I want to do it during runtime of the program, loading the information regarding the msgs, because i can have different projects where the information for different msg IDs can differ.
I've a non-C file, where basically all the information is written. Lets stay frame named
GetStatus{
bit 0 - 7 width
bit 8 - 15 height
.
.
.
}
etc.
How to read it on runtime and decode the messages? On runtime I'm not able to create variables and structures anymore!
Any advices? Is some sort of scripting language the solution? Which, how? :-)
Thanks in advance for all replays!
Re: "Creating variable" on runtime? - message decoding
I don't have a ready-to-go solution for you but unless your data is very complicated it's shouldn't be that hard to create a file that contains descriptions for all the specific messages in a project and then use that info to process the data in runtime.
The description for one specific message migth for instance be a vector containing the description for each field within the 8 bytes.
Re: "Creating variable" on runtime? - message decoding
Thanks for the reply, S_M_A.
Actually we have some sort of file with the frame descriptions, but imagine this:
You receive a message, according to the ID you identify the frame. So you have to go through the file, find the appropriate frame, get all the information, decode the 8 bytes and print it out.
You need to repeat this process for "every" received message. This solution is straightforward, but the effectiveness is low and computation time high in my opinion.
I was thinking about some sort of solution where you open this description file and "pre-decode" the frames -> store/create the structures for every message. During the reading you assign the received data into the structure and return the bits you want.
something like:
typedef struct data_bytes
{
BYTE byte1;
BYTE byte2;
BYTE byte3;
.
.
.
} DATA_BYTES;
typedef struct data_bits
{
width :6
height 6
length : 12
.
.
.
} DATA_BITS;
typedef union
{
DATA_BITS;
DATA_BYTES;
} DATA
DATA.DATA_BITS.width
DATA.DATA_BITS.height
This code is just for example. The approach will be maybe totally different. Because we have the data in an array, so there has to be some pre-processing.
Re: "Creating variable" on runtime? - message decoding
there are many ways of doing such a thing, ranging from the straightforward ( S_M_A's suggestion; BTW, I think he already implied the preprocessing and caching of the file data, no need to read the file for each message ) to the impossible ( if you want to access those structs as if they were variables of the corresponding types then there's no way of doing it at runtime being c++ a statically typed language with no runtime reflection support, which is a very good thing BTW :) ) depending on your requirements.
so, how are you going to access the data ? how is the [id]<->[byte interpretation] mapping supposed to work ? how much general the solution should be ( just POD's with bitfields, or general classes, is this C or C++ ) ? etc ...
supposing this is C, a straightforward solution could be
Code:
// raw message
typedef struct
{
ULONG timestamp;
BYTE ID;
BYTE data_length;
BYTE data[8];
} MSG_FRAME;
typedef void (MSG_ENCODER*)( BYTE* );
typedef struct
{
char* guid;
MSG_ENCODER* encoder;
} MSG_DESCRIPTOR;
// messages
typedef struct
{
ULONG width: 6;
// ...
} MSG_A;
char* MSG_GUID_A = "28d4604a-c084-4943-8460-fc0753b6ed3e";
void MSG_ENCODER_A( BYTE* data )
{
MSG_A* msg = (MSG_A*) data; // of course, you have to make sure that this is actually portable
// access msg->width, etc ...
}
//...
int main()
{
// populate the message mapping
MSG_DESCRIPTOR message_descriptor_map[] = {
{ MSG_GUID_A, MSG_ENCODER_A }
/*, ... */ };
MSG_ENCODER message_map[ 256 ];
// here, say, open a file containing an ordered list of GUIDs; for each GUID look up the corresponding encoder
// in message_descriptor_map and store it in message_map at the current position; in this way, you can control
// the ID mapping at runtime ...
// pump messages
while( /* message loop */ )
{
MSG_FRAME message = /* some message source */;
(*message_map[ message.ID ])( message.data );
}
}
Re: "Creating variable" on runtime? - message decoding
Thanks for your reply, superbonzo, but your code snippet is not applicable. We have to separate the problem into two sections.
1) the application itsefl, which is able just to print out some text - bus monitor. This app is already compiled and ready to go.
2) External data file, which describes the message frames I receive via the bus and their internal structure like:
GetStatus: 32{
width :6
height :6
.
.
.
}
Because this internal structure is project dependant, the structure differ. So for another project the GetStatus frame can be:
GetStatus: 32{
positionx :12
positiony :12
.
.
.
}
Thats why I can't define any structures before, I need to do this during runtime of the app.
Do you know CANoe or CANalyzer from VECTOR? This is lets say something, what I want to do in a simpler way. To track the LIN bus.
So I need somehow pre-preprocess the message frames during runtime, so it does not take a lot of time computing during printing.
Re: "Creating variable" on runtime? - message decoding
Quote:
Originally Posted by
HellMaster22
Thats why I can't define any structures before, I need to do this during runtime of the app.
What you need is to come up with a class that properly describes the data, instead of attempting to associate a specific C/C++ struct with the data. In other words, a "data independent class".
For example, why not do this:
Code:
#include <map>
#include <string>
typedef std::map<std::string, int> StringToValueMap;
class StructureDefinition
{
StringToValueMap m_sValueMap;
//...
// maybe other members describing the data
};
Now, that map will be populated with your information. The strings would be for example "width" and "height", and the value in the map would be 6 and 10 (or whatever). For another data, you would have strings "positionx" and "positiony", along with their respective values.
Then you know
1) The number of entries
2) Each entry name
3) Each entry's value
4) The sum of the number of bits (you need to loop through the map and do a simple sum of the value entry).
So now you have all the information you need, and your program's job is to now interpret this information.
Regards,
Paul McKenzie
Re: "Creating variable" on runtime? - message decoding
Hi Paul,
thank you very much for your reply. I'm not a C++ programmer. Can you be more specific regarding your example?
Lets say you receive on the bus the following message:
ID 10
Length 4 bytes
Data = {0A, F3, 81, 15} hexa
In the description file there is this information:
Code:
MessageFrames {
GetStatus, 10, 4 (first number is ID, second number of bytes)
{
width, 0 (numbers represent bit offset)
height, 6
posx, 12
posy, 18
range, 24
}
other_frame1, 15, 2
{
.
.
.
}
other_frame2, 16, 8
{
.
.
.
}
}
Meanwhile I'll look on the internet how does the mapping work...
Re: "Creating variable" on runtime? - message decoding
Oh! I didn't realize that this was LIN. Why not just buy the LIN option from Vector? Yes I know it's not cheap but on the other hand every hour you put into it cost as well. This is an area where I have a lot of experience and I've seen many companies put a lot of effort into trying to make their own thing just to find out that it's not good enough.
Re: "Creating variable" on runtime? - message decoding
You hit the nail S_M_A. We have a CANoe from VECTOR, but as you mentioned, its not really the cheapest one... and for some simpler tasks, where someone just needs to track the bus, its to complicated to use, and its licensed. So we wanted to make some simpler solution. But thats not the question. :-)
The question stays. How to decode the messages defined in an external file (LIN description file) on runtime. :-) Use some scripting language? Which and why? Or some other approach? There has to be some solution. And not reading the file at runtime and getting the information from in on fly for every received message. :-)
Re: "Creating variable" on runtime? - message decoding
Quote:
Originally Posted by
HellMaster22
Thanks for your reply, superbonzo, but your code snippet is not applicable.
sorry, so, by "project dependant" you mean that the data structure itself is choosen at runtime, not just the id mapping, don't you ?
if yes, then, as suggested by Paul, you just need to define the proper abtraction for your data layout, which depends on the generality of the data structures you want to support. If they're just POD bitfields as it seems, then assuming a LIN file has the structure shown in your post ( sorry, I don't know VECTOR ) you could write something like:
Code:
// #include ... MSG_FRAME, BYTE, ...
#include <iostream>
#include <string>
#include <vector>
typedef BYTE Id;
struct BitField
{
std::size_t size; // in bit
std::string name;
};
class Frame
{
public:
Id GetId() const { return id_; }
friend std::istream& operator>>( std::istream& is, Frame& frame )
{
char d; // delimiter dummy
// parse header
is >> name_ >> d >> id_ >> d >> size_; // ignoring the '(...)' comments
is >> d; /// '{'
// parse bit fields
BitField field;
while( is >> field.name >> c >> field.size )
{
fields_.push_back( field );
}
// parse foot
is >> d; /// '}'
return is;
}
class View
{
public:
View( Frame const& frame, MSG_FRAME const& msg )
: frame_( frame ), msg_( msg ) {}
friend std::ostream& operator<<( std::ostream& os, View const& view )
{
assert( view.frame_.id_ == view.msg_.ID );
os << view.frame_.name_ // << .. put whatever info you like: size, id etc ...
<< std::endl;
// then the single fields
for( BitField const& field: view.frame_.fields_ )
{
os << field.name << " = " << // perform some bit twiddling to extract the field value from view.msg_.data
<< std::endl;
}
return os;
}
private:
Frame const& frame_;
MSG_FRAME const& msg_;
};
private:
Id id_;
std::size_t size_; // in bytes
std::string name_;
std::vector<BitField> fields_;
};
Frame::View MakeView( Frame const& frame, MSG_FRAME const& msg )
{
return Frame::View( frame, msg );
}
int main()
{
std::unordered_map<Id,Frame> message_map;
{
// populate the message map
std::ifstream message_file( /*... your 'external' file ... */ );
// skip the first "MessageFrames {" line
message_file.ignore(std::numeric_limits<std::streamsize>::max(), '\n')
// then go
Frame running_frame;
while( message_file >> running_frame )
{
message_map[ running_frame.GetId() ] = running_frame;
}
}
// pump messages
MSG_FRAME msg = /* some message source */;
std::cout << MakeView( message_map[ msg.ID ], msg );
}
BTW, note that due to the use of <iostream>, approaches like this result quite flexible, especially if combined with other iostream-aware libraries ( take a look at boost for example ) ...
Re: "Creating variable" on runtime? - message decoding
Quote:
Originally Posted by
superbonzo
sorry, so, by "project dependant" you mean that the data structure itself is choosen at runtime, not just the id mapping, don't you ?
Seems to me, I've described the whole problem wrong. :-/ Sorry for that.
On the bus you transmit and receive LIN messages in this simplified form:
ID (1byte), Data (2 - 4 - 8 bytes lenght differs)
You read these messages with a C API from National Instruments one after another or all at once (not important) and store these data in a defined struct like this:
Code:
struct
{
ID;
timestamp;
data_length;
UINT8 data[8]; // Frame
}
Example: ID = 48; timestamp = 5000; data_length = 4; data = 0A 25 3C 10 (hex) => data[8] = {10, 37, 60, 10, 0, 0, 0, 0}
Then I've an external ldf file (LIN description file) which describes the inner structure of the received data (This file has nothing to do with the NI LIN card):
Part of the ldf file looks like this:
Code:
Frames {
BCM_LIN5_A : 48, BCM_Gp, 4 { (message name, message ID, not important info now, length of message in bytes)
UserSetMinute, 0; (sub-information, bit offset - this parameter starts at bit 0)
UserSetSecond, 8; (sub-information, bit offset - this parameter starts at bit 8)
UserSetHour, 16;
CarModeL5, 24;
}
BCM_LIN5_C : 55, BCM_Gp, 8 {
RHSBLSlewRateRequest, 0;
RHFrontSMRequest, 4;
LHFrontSMRequest, 6;
RHSBLPWMDutyRequest, 8;
RHDRLPosSlewRateRequest, 16;
LHSBLSlewRateRequest, 20;
RHDRLPosPWMDutyReq, 24;
LHDRLPosSlewRateReq, 32;
LHDRLPosPWMDutyReq, 40;
LHSBLPWMDutyRequest, 48;
}
CLOCK_A : 49, CLOCK, 4 {
CLKHWPartNo, 0;
CLKFaultStatus, 16;
.
.
.
} // End of Frame description
So from our example,
UserSetMinute = 16 (0x10)
UserSetSecond = 60 (0x3C)
UserSetHour = 37 (0x25)
CarModeL5 = 10 (0x0A)
The problem is, this frame description can differ from one project to another. So next time ID 48 can be message called GetParam, the length can be 2 bytes and there can be 8 informations (2 bits for every info).
As I mentioned, I'm really weak in C++. So I'll study your code. Thank you very much for your effort.
I'm looking forward your next comments, advices etc. :-)
Re: "Creating variable" on runtime? - message decoding
yes, my tentative code snippet is supposed to do exactly what you describe: it parses the description file extracting each frame definition, which is then put in an unordered_map that allows you to retrieve the frame given an ID ( in amortized constant time, BTW ). Then, the frame object and the message are given to a "view" object that implements the actual printing of the message interpreted as the corresponding frame ...
... it's just not complete as it lacks proper parsing of the file, the actual value extraction, possibly a better overall design and a few typos.
Quote:
Originally Posted by
HellMaster22
store these data in a defined struct like this:
[...]
Part of the ldf file looks like this:
[...]
I'm looking forward your next comments, advices etc. :-)
well, you need to specify the exact file format and the exact data message structure in order to implement the parsing and the extraction code.
Quote:
Originally Posted by
HellMaster22
As I mentioned, I'm really weak in C++
are you looking for a C-only solution ?
Re: "Creating variable" on runtime? - message decoding
Thanks for clarifying superbonzo. If I understand it right, the map class is for parsing the description file. First I need to study the class more in depth.
But the C solution would be easier for me I guess. I've decided to create an array[64] of structs. Each struct will contain the name of message, number of sub-informations and a linked list. This list will contain name of subinfo, bit offset and a mask.
Any thoughts? :-)
Re: "Creating variable" on runtime? - message decoding
Quote:
Originally Posted by
HellMaster22
But the C solution would be easier for me I guess. I've decided to create an array[64] of structs. Each struct will contain the name of message, number of sub-informations and a linked list.
My suggestion is to go to C++. What you describe makes C++ to be an easier solution than 'C'.
For example, std::list<T> is the standard linked list class for C++. To have a key/value map, there is std::map, etc. There are no such classes for 'C', as you have to code all of that yourself.
I think you need to think of it at a higher abstraction instead of getting down to what C or C++ types to use. Whether you're weak in C++ or not, that isn't important. What is important is that you can envision the design and architecture, independent of what language you are going to use.
You have information, and you want to process this information in a dynamic fashion. That is all that this all boils down to, at least IMO. How does, say, an imaging program work with various images, not knowing beforehand the number of bits-per-pixel, whether it is gray-scale, B/W, color, CMYK, etc.? Isn't that information also varying, depending on the "project" (in this case a "project" would be "image file")? Somehow, those imaging programs seem to work.
The good thing about C++ is that higher abstractions can be coded more easily than for C. You want a key/value lookup, then you have std::map. You want a linked list, you have std::list. You want a dynamic array, you have std::vector, etc. In other words, using the C++ library gives you ready made components so that you can put together your design, similar to a jigsaw puzzle and have a working program with less bugs than a 'C' program. You just can't do that easily with 'C' as you have to code your own linked lists, maps, dynamic arrays (and then make sure there are no bugs in doing so), etc.
Regards,
Paul McKenzie