policy based design of "modern c++ design"

[PredicateNormative]

Popular on those famous libraries, but not so popular of my environment.
Atleast there are a lot of the jobs would require the programmers should have
strong knowledge about OOP but not GP, most of the jobs not even mention
about GP at all.

When I ask them why, some of them would say "because it is only suit for small projects and simple cases, STL and boost are small, and they are designed to solve for those simple case, they are not big and complicated enough, GP can't handle those complex large scale project"

Whoever has said the above to you is missing the point of template programming and has made an incorrect and misguided statement. Many large projects use a combination of OOP and templates in harmony to solve complex problems. Generally, templates are not a complete solution, but can used be a very powerful means to an end.

In addition, the person who made the above statement has little understanding of scalable architecture. The reason why I state this is that at the end of the day, a properly architected large project can be broken down into a hierarchy of smaller resuable sub-components. Thus, it can be argued that large projects are made of many smaller (simple) projects. Thus, the argument that "because [GP] is only suit for small projects and simple cases, STL and boost are small, and they are designed to solve for those simple case, they are not big and complicated enough, GP can't handle those complex large scale project" clearly misses the point of how larger projects are constructed.

[stereoMatching]

I've worked for two very large companies, and in both companies, those who have a good understanding of how to write templates tend to be few and far between.

Do that means I am learning a powerful design philosophy but can't be used or better don't
use it in the real case?

I was finding a way to produce better abstraction codes without any performance degrade, when
I found GP and TMP, they make me very excited.

a programmer needs to have a good understanding of both OOP and templates

Looks like I have some mistakes about policy based design if policy based design is so difficult.
I am not very good at OOP or template, only know the basic ideas of them and study a few of
design patterns. Please correct me if I get the wrong idea of it.

For me, policy based design only try to composite different strategies(could be everything) into
a body, you don't need to(but could) inherits those policy(or policies)
you could just composite them(treat them as private data member) to generate
different types or even apply those policy(s) on function.

Each of them have their own advantages and weakness, the main idea is extract those "polymorphic"
actions from the body and make them become orthogonal policy(s) as best as you can.
The idea is a little bit different than the "is a" relationship of OOP(more like "has a" or "is
implemented by") when you try to combine those policy(s). It is a small "compiler" which help you
generate codes same as those handcrafted codes.

[nuzzle]

1) GP is used to write "patterns" in the form of generic components, so you would essentially need a theory of patterns of patterns; even with such a theory, program designers would continue studying how to make programs, not how to make patterns to make programs.

2) OOP places a partial relation on types, GP works with heterogeneous arbitrary relations on types; therefore, such a theory would be necessarely much more complex then the OO theory, making it suitable to experts in any case. BTW, if you think about it, the more powerful concept programming is the more it will come near to the general non-programming related idea of "concept" ... therefore, such a general theory of concept design would be applicable to general problem solving or even philosophy as well, isn't this asking too much ?

I don't think it's as hard as you make it. OOP and GP are based on two different kinds of polymorphism namely inclusion (or subtype) polymorphism and parametric polymorphism respectively. The latter is sometimes also called generics hence GP (and templates is a static form of generics).

Now what programming paradigm corresponds best to parametric polymorphism? That's right, it's functional programming. So a theory of GP design must approach program design from a functional point of view. It must convey a view of the world based on functional concepts.

This thread has helped me organize my thought on this matter and I've somewhat changed my mind. I now think that GP may be an alternative to OOP as a design methology for C++ if it's approached in the broader context of functional programming and not limited to just templates.

[PredicateNormative]

Do that means I am learning a powerful design philosophy but can't be used or better don't
use it in the real case?

No, all it means is that the majority of programmers will not be able to easily maintain the code - that does not necessarily mean you will be discouraged from writing templates and taking advantage of policy based design. All I am saying is, that the fact that most programmers will not be able to maintain that code is the reason why companies in general do not encourage template programming.

The company I currently work for does not generally encourage template programming, however, it is very happy for complex templates to be written where there is good justification as long as they are well documented.

I was finding a way to produce better abstraction codes without any performance degrade, when
I found GP and TMP, they make me very excited.

That is where templates can really come into their own. I use policy based design and TMP in code that I develop for a number of in-house libraries. The end result is that I can push a large number of decisions down to compile-time, and thus levy a good run-time performance benefit. However, I would give a word of caution, if you are writing code for a company, then I would advise that you don't get too elaborate with TMP - a little bit is fine, but it should be well documented and well justified. In addition, avoid complex template declarations on classes that you intend to be used by the user of your library; firstly, it will scare prospective users away, and secondly it is good practice to ensure that libraries are easy to use, even if that means that making the interface easy makes the library harder to write.

Looks like I have some mistakes about policy based design if policy based design is so difficult.
I am not very good at OOP or template, only know the basic ideas of them and study a few of
design patterns. Please correct me if I get the wrong idea of it.

Only having a basic understanding of OOP and templates should not stop you from getting into policy based design. As you make mistakes, then out of necessity you will learn a lot, both in respect to object orientation and template programming.

[PredicateNormative]

Here is a reason, why a lot of companies will not encourage their employees to write non-trivial templates. If you ever get into policy based design with a bit of MTP mixed in, then expect to get some relatively crazy template errors. This is a single error from some of my code that I am developing (my own personal library that I was considering open sourcing at one point - hence the name). There are plenty more errors of equal length generated (61 to be precise) as a result of the root of this error. Although the last few words of the error state what the reason is, they don't state the cause... that's hopefully wrapped up in the three pages of trace output for the error. What I mean by "it doesn't state the cause" is that there is an appropriate operator+ function defined (at least as far as intention goes), but it is probably being messed up by an incorrect typedef somewhere. The Microsoft compiler fails on the same line, but it expresses the error in a different way, stating that it cannot deduce template arguments and attempts to match against types that shouldn't even come into the equation (around different 100 types, about 90 of which are from the STL which is a little strange). Anyway, it's always good to have more than one compiler at hand when something like this happens, since sometimes one tracks the error down in a more concise manner than the other (although in this case both are pretty verbose)!

/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/image/assignable_pixel.hpp:  In function ‘void otl::ip::detail::image::assign_iterators(Iterator, 
otl::ip::detail::image::data_type::configuration_category::interleaved,  const otl::ip::detail::dimensions&,  otl::ip::detail::image::data_type::triplet_block, 
Iterator2&, Iterator2&, Iterator2&) [with Iterator =  otl::detail::random_access_iterator<const unsigned char>,  Iterator2 = otl::iterator::pattern_iterator<
otl::detail::random_access_iterator<const unsigned char>, 1u,  3>, otl::ip::detail::image::data_type::triplet_block =  otl::ip::detail::image::data_type::block<
3u, 1u, 3u, 3u, otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>]’:
/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/image/pixel_iterator_impl.hpp:195:7:    instantiated from ‘otl::ip::detail::image::pixel_iterator_impl<
Iterator, DataType,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>::pixel_iterator_impl(Iterator,  const otl::ip::detail::dimensions&) 
[with Iterator = otl::detail::random_access_iterator<const unsigned  char>, DataType =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved> >]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/image/pixel_iterator.hpp:48:48:    instantiated from  ‘otl::ip::detail::image::pixel_iterator<Iterator, 
DataType>::pixel_iterator(Iterator, const  otl::ip::detail::dimensions&) [with Iterator =  otl::detail::random_access_iterator<const unsigned char>, DataType  = 
otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u,  1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>  >]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/image.hpp:93:97:    instantiated from ‘otl::ip::image<T, DataType,  Container>::const_pixel_iterator otl::ip::image<T, DataType, 
Container>::pixel_begin() const [with T = unsigned char, DataType =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved>  >, Container = otl::buffer<unsigned char>,  otl::ip::image<T, DataType, Container>::const_pixel_iterator 
=  otl::ip::detail::image::pixel_iterator<otl::detail::random_access_iterator<const  unsigned char>, otl::ip::detail::image::data_type::rgb_type<
otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>  > >, 
typename otl::ip::detail::image::header<DataType>::data_type =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved>  >, typename Container::const_iterator =  otl::detail::random_access_iterator<const unsigned char>]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/algorithm/detail/cpu_colour_conversion/change_configuration.hpp:24:8:    instantiated from 
‘static void  otl::ip::algorithm::detail::cpu_colour_conversion::change_configuration::run(const  otl::ip::image<Ty1, BaseDataType<ConfigurationCategory1>,  Container1>&, 
otl::ip::image<Ty2, BaseDataType<ConfigurationCategory2>,  Container2>&) [with Ty1 = unsigned char, Ty2 = unsigned char,  ConfigurationCategory1 = 
otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>,  
ConfigurationCategory2 =  otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u,  1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>,  Container1 = 
otl::buffer<unsigned char>, Container2 = otl::buffer<unsigned  char>, BaseDataType = otl::ip::detail::image::data_type::rgb_type]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/algorithm/detail/translate.hpp:312:5:    instantiated from ‘void otl::ip::algorithm::detail::translate_impl(
const otl::ip::image<Ty1, BaseDataType<ConfigurationCategory1>,  Container1>&, otl::ip::image<Ty2,  BaseDataType<ConfigurationCategory2>, Container2>&) 
[with ConversionDefinition = otl::ip::cpu_colour_conversion, Ty1 =  unsigned char, ConfigurationCategory1 =  otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved>,  Container1 = otl::buffer<unsigned char>, Ty2 = unsigned char,  ConfigurationCategory2 = 
otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>,  
Container2 = otl::buffer<unsigned char>, BaseDataType = otl::ip::detail::image::data_type::rgb_type]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/algorithm/translate.hpp:15:5:    instantiated from ‘void otl::ip::algorithm::translate(const  SourceImageType&, 
DestinationImageType&) [with ConversionDefinition =  otl::ip::cpu_colour_conversion, SourceImageType =  otl::ip::image<unsigned char, 
otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u,  1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>  > >, 
DestinationImageType = otl::ip::image<unsigned char,  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved> > >]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/file/pnm.hpp:167:7:    instantiated from ‘static void  otl::ip::detail::file::pnm<ConversionDefinition>::load_data_(
std::istream&, const otl::ip::detail::file::pnm_detail::header&,  otl::ip::image<T, DataType, Container>&) [with  DestinationValueType = unsigned char, DestinationDataType = 
otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u,  1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>  >, 
T = unsigned char, DataType =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<
3u, 1u, 3u, 3u, otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>, 
otl::ip::detail::image::data_type::configuration_category::interleaved>  >, Container = otl::buffer<unsigned char>,  ConversionDefinition = otl::ip::cpu_colour_conversion, 
std::istream = std::basic_istream<char>]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/file/pnm.hpp:153:9:    instantiated from ‘static void  otl::ip::detail::file::pnm<ConversionDefinition>::load_(
std::istream&, const otl::ip::detail::file::pnm_detail::header&,  otl::ip::image<T, DataType, Container>&) [with  DestinationDataType = 
otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u,  1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>  >, 
T = unsigned char, DataType =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<
3u, 1u, 3u, 3u, otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>, 
otl::ip::detail::image::data_type::configuration_category::interleaved>  >, Container = otl::buffer<unsigned char>,  ConversionDefinition = otl::ip::cpu_colour_conversion, 
std::istream = std::basic_istream<char>]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/file/pnm.hpp:106:9:    instantiated from ‘static void  otl::ip::detail::file::pnm<ConversionDefinition>::load(
const std::string&, otl::ip::image<T, DataType,  Container>&) [with T = unsigned char, DataType =  otl::ip::detail::image::data_type::rgb_type<
otl::ip::detail::image::data_type::data_layout<otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u, 
otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,   otl::ip::detail::image::data_type::configuration_category::interleaved>  >, 
Container = otl::buffer<unsigned char>, ConversionDefinition =  otl::ip::cpu_colour_conversion, std::string =  std::basic_string<char>]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/file.hpp:63:7:    instantiated from ‘static void  otl::ip::file<ConversionDefinition>::load_pnm(const  std::string&, 
otl::ip::image<T, DataType, Container>&) [with T = unsigned  char, DataType =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved>  >, Container = otl::buffer<unsigned char>,  ConversionDefinition = otl::ip::cpu_colour_conversion, 
std::string = std::basic_string<char>]’
/home/andrew/projects/OpenTemplateLibrary/otl/ip/file.hpp:36:33:    instantiated from ‘static void  otl::ip::file<ConversionDefinition>::load(const std::string&, 
otl::ip::image<T, DataType, Container>&) [with T = unsigned  char, DataType =  otl::ip::detail::image::data_type::rgb_type<otl::ip::detail::image::data_type::data_layout<
otl::ip::detail::image::data_type::block<3u, 1u, 3u, 3u,  otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>,  
otl::ip::detail::image::data_type::configuration_category::interleaved>  >, Container = otl::buffer<unsigned char>,  ConversionDefinition = otl::ip::cpu_colour_conversion, 
std::string = std::basic_string<char>]’
/home/andrew/.codelite/workspace/main.cpp:11:97:   instantiated from here
/home/andrew/projects/OpenTemplateLibrary/otl/ip/detail/image/assignable_pixel.hpp:149:5:  error: no match for ‘operator+’ in ‘position + 1’

I guess it'll probably take me a while to sift through the error to figure out what's wrong with the code, but that's life. However, there is a point that I would like to make, and that is, that although an error like the above looks crazy; by the time you remove all of the namespace information and format it into a more manageable form, it's generally pretty easy to follow to figure out what the compiler is complaining about. Therefore, the point that I would like to make, is that if you get into template programming and see an error like the above, don't be put off - it's probably not all that bad.

[D_Drmmr]

What I really don't understand is why such error messages always seem to come in the form 'error - instantiated from - ... - instantiated from - my code'. At least 99% of the time I (as a programmer) am interested only in the last part or last few parts. If the order of the parts was simply reversed in the compiler output, figuring out what such an error means would be so much easier.

[superbonzo]

I agree, a reversed template "call stack" would be helpful; anyway, I think it's more an IDE related issue; the IDE should be responsible of processing the compiler output graphically in a way similar to how the debugger shows the runtime state of variables.
In this way, the compiler would be free of outputting all the information it has about a failing template instantiation ( as now stands, often is too much, sometime is too little ) and the IDE would present it in a nice graphical way, with full on demand info on the whole instantiation tree.

Another issue is at the language support level, it's just too easy to invoke compiler bugs when working with templates ... ( at least in vc++ )

[stereoMatching]

Anyway, it's always good to have more than one compiler at hand when something like this happens, since sometimes one tracks the error down in a more concise manner than the other

I find out that the error messages of gcc4.6.2 is much more easy to understand than the error
messages generated by vc2010.

by the time you remove all of the namespace information and format it into a more manageable form, it's generally pretty easy to follow to figure out what the compiler is complaining about.

I hope so, this technique is mentioned by item49 of "effective stl", looks like I have to take a look
at it since I always play with template

[PredicateNormative]

The thing that I would like to see is for the compiler output not to simply be displayed as plain text. For example it would be nice if:

In function ‘void otl::ip::detail::image::assign_iterators(Iterator, 
otl::ip::detail::image::data_type::configuration_category::interleaved,  const otl::ip::detail::dimensions&,  otl::ip::detail::image::data_type::triplet_block, 
Iterator2&, Iterator2&, Iterator2&) [with Iterator =  otl::detail::random_access_iterator<const unsigned char>,  Iterator2 = otl::iterator::pattern_iterator<
otl::detail::random_access_iterator<const unsigned char>, 1u,  3>, otl::ip::detail::image::data_type::triplet_block =  otl::ip::detail::image::data_type::block<
3u, 1u, 3u, 3u, otl::ip::detail::image::data_type::data_configuration_description::three_plane_data>]’:

was displayed like:

In function 

‘void assign_iterators(Iterator, interleaved,   const dimensions&,  triplet_block, Iterator2&, Iterator2&, Iterator2&) 

[with 
  Iterator =  random_access_iterator<const unsigned char>,     
  Iterator2 = pattern_iterator<random_access_iterator<const unsigned char>, 1u,   3>, 
  triplet_block =  block<3u, 1u, 3u, 3u, three_plane_data>
]’:

where you could hover the mouse pointer over each item and it would tell you the namespace information about it.

I also agree that a reverse stack for each error might be helpful.

[Paul McKenzie]

The thing that I would like to see is for the compiler output not to simply be displayed as plain text. For example it would be nice if:

I believe there was a program that simplified the compiler output for template errors. Don't remember the name now, but I definitely know it existed.

Don't hold me to it, but I believe it was called C++ Filt or something like that.

Regards,

Paul McKenzie

[JohnW@Wessex]

I've worked for two very large companies, and in both companies, those who have a good understanding of how to write templates tend to be few and far between.

Very true! I've found that of those that don't understand, some will pick it up very quickly, while others never quite seem to grasp it.

I've used a lot of template programming in the last few years, but almost all of it was in the implementation of an internal common library. For the users of the library (the other coders in the team), there was very little template knowledge required to use it, except for the very occasional template parameter to help the compiler resolve an abiguity. It was only if someone were to want to modify or add to the library would an in-depth knowledge of templates be required.

[stereoMatching]

Very true! I've found that of those that don't understand, some will pick it up very quickly, while others never quite seem to grasp it.

Maybe that is why so many programmers don't believe template could give a hand on balancing
between performance, robustness and abstraction.

Looks like C++ has to find a better way to make template become more instinct
Atleast it should make the users have a better feeling about "compile time cost" and "run time cost"

[Peter_B]

I believe there was a program that simplified the compiler output for template errors. Don't remember the name now, but I definitely know it existed.

Don't hold me to it, but I believe it was called C++ Filt or something like that.

Regards,

Paul McKenzie

You may be thinking of stlfilt (http://www.bdsoft.com/tools/stlfilt.html). I've never used it but it looks quite useful.

[PredicateNormative]

I believe there was a program that simplified the compiler output for template errors. Don't remember the name now, but I definitely know it existed.

Don't hold me to it, but I believe it was called C++ Filt or something like that.

Regards,

Paul McKenzie

I have heard that such things exist, but I've never tried one... given the amount of template code I write I'm beginning to think that it would be a good thing. Thanks for the advice.

[PredicateNormative]

You may be thinking of stlfilt (http://www.bdsoft.com/tools/stlfilt.html). I've never used it but it looks quite useful.

Thanks for the link, I'll take a look.