InfoQ: Model-Driven Development: Where are the Successes?

I found a good blog post about more experiences onmodel-driven deleopment at InfoQ: Model-Driven Development: Where are the Successes?. Read it and the articles mentioned.

I have also read through a summary of the study of Sascha Kirstan which was very interesting.Unfotrunatley i don't know if it is avaibale on thenet yet. However both he and Jon Whittle clearly stated that in practice the advantages with model based develoipment are not so much efficiency as quality. In practice the efficiency gains tend to be around 20-30%, not 3-5 times as one can hear among certain the evangelists. And the quality gains are in the same order.
But reducing the number of faults and errors with 20% impresses me more than reducing development costs with 20%.

What is a model?

My previous post got me thinking about what a model is and why we model. I think everybody agrees that a text specification written in Word is not a model, while something in UML is.

The Cambridge Advanced Learner's Dictionary defines a model as

A representation of something, either as a physical object which is usually smaller than the real object, or as a simple description of the object which might be used in calculations.

I like the informal definition I heard from Jon Whittle last week: A model is an abstraction of the real thing for a specific purpose. However with this definition even a specification document written in Word will be a model and I don't think that makes sense.
Mellor et al. defines a model (Ludewig defines even more formal criteria):

A model is a coherent set of formal elements describing something (for example, a system, bank, phone, or train) built for some purpose that is amenable to a particular form of analysis, such as...

With this definition it is quite easy to exclude a Word document as a model, since it isn't composed of formal elements. It can incorporate diagrams based on a model, e.g. a class diagram from an UML model, but not the model itself.

In addition to this there should be a purpose with the model. John Daniels identifies three different types of models, each with a different purpose:

• Conceptual model
• Specification model
• Implementation model

When reading the article it is obvious that just by stating one has a UML model or a Simulink model really does not say anything about the purpose. What I am curios about is if one is using the same model for different purposes? Is it possible to use a model for completely different purposes with any degree of efficiency and quality? I am not talking about a transformation from one model to another, for example code generation or MDA. Do you use the same Simulink model for both conceptually understand the problem, specify properties of the solution and describe the implementation?

I had great help from my colleague Niklas Mellegård in his licentiate thesis Method and Tool Support for Automotive Software Engineering, which is defended on Thursday 30 September at 13:00 in room Torg 2, Patricia building, Lindholmen. Discussion leader will be Prof. Martin Törngren, Dept of Machine Design, Royal Institute of Technology.

Software quality attributes

I got the following e-mail from a student after discussing definitions and frameworks of quality attributes:

Hello,

It seems there are 2 main groups of ISO standards dealing with software quality:

1. a 4 part ISO 9126 standard (QA considered are reliability, usability, efficiency, maintainability, portability AND, actually, functionality)
2. second generation of ISO software quality standards referred to as SQuaRE (Software product Quality Requirements and Evaluation): ISO/IEC 25000:2005, ISO/IEC TR 25021:2007 and ISO/IEC TR 25060:2010.

I couldn't find any free copies of these standards, so we can't really use them.

Anyway, it would be interesting to get during the lectures some additional information about different quality attributes models out there (early McCall and Boehm models, FURPS, ISO models etc.).

I replied with this:

You are correct. You can't access the ISO standards (without paying).

• For a very soft introduction to quality attributes, read this: http://www.stellman-greene.com/2009/10/03/using-nonfunctional-requirements/
• An in-depth analysis of a few quality attributes are available at http://www.sei.cmu.edu/library/abstracts/reports/95tr021.cfm
• For a definition of terminology I suggest IEEE std 610.12, bit this covers much(!) more than just quality attributes. It is available at http://ieeexplore.ieee.org/
• You can also read what the PhD students at Blekinge Technical University wrote down in a course about software quality: http://www.bth.se/tek/besq.nsf/pages/017bd879b7c9165dc12570680047aae2!OpenDocument
• And as usual, wikipedia is a good introduction, look at "software quality" or "FURPS" for example.

I think it will be a little too much to go into detail about different quality frameworks in this course, above they dedicated an entire course at PhD level to the subject.

Hope this helps,

Ulrik

Do you use model based development?

I heard an interesting keynote speech from Jon Whittle this morning with the title The Truth About Model-Driven Development: Who's doing it, how and why?
He presented some findings on experiences from using model-based development in industry from the EA-MDE project. I talked to him afterwards and there apparently had only been one automotive company in the study, and I personally think that the conclusions might have been different if they were more data from them. If you are working model-based in your company and want to participate in the study you can do so by answering the questionare on the project web site.
One benefit of model-based development which I expected to see in the study was that it allows domain experts to actually do implementation, for example can chassis control experts actually do code for use in ECUs.
Note: You can substitue model-based development with several other acronyms and Jon's study is still relevant; MDD, MBSE, MDA, ...

I find this study to be similar to what Sascha Kirstan is working on in the automotive industry. I participated in Sascha's study this spring and just got a report with preliminary results, which I haven't had the time to read yet. I sure hope these two researcher will look at each others result and see if they can corroborate their findings or if they are contradictions.

Jon concluded his talk with the top ten tips for companies wanting to adopt model-based development and here they are (but to truly utilise them you must know a lot more about the context where they were found):

1. Keep the domains (modelled, I assume) tight and narrow.
2. Target well known domains.
3. Put MDD on the critical path (he means that pilot projects never get sufficient attention and resources).
4. MDD works best form the ground up.
5. Be careful of gains that are offset elsewhere.
6. Don't obsess about code generation.
7. Not everyone can think abstractly.
8. Most projects fail at scale-up
9. Match tolls and processes to the way people think, not the other way around
10. Ok, there was only 9...

Why I don't like formal methods...

I know the title of this port is a bit controversial and with it I alienate a lot of researchers, even at my own department. But I will try to clarify my arguments and hope that someone can prove them to be wrong.

• To be more precise; it is not the formal methods in itself I dislike, but the prominence they seem to have at prestigious software engineering conferences. I think it is not at all in proportion to how well-used formal methods are among professional development.
• Formal methods are really attractive from a researchers perspective. You can use concepts as theorem, proof and deduction and other nice things. But nice is not the same as relevant.
• Even though I have heard about formal methods as one of the enablers to establish software engineering as a "real" engineering discipline for almost ten years I still don't think they are nearer being well-established now than then.
• Some claim that you can never be sure of the software unless you can prove properties about it, and I agree that presently formal methods are the only technique that promises to do so. But there is a lot of successful software out there which have not been proven.
• Specifications that fulfil the requirement of being interpreted formally are hard to write. Compare it to learn a new programming language and be proficient enough to use it without any side effects.
• I don't know if formal methods scale well. It is one thing to use it on a nice prototype system with 30 entities developed in your lab. It is another thing to use it on a system with 500 entities, many of these with quite varying quality in requirements and code.
• I still don't know of any that uses formal methods for real; i.e. as part of the normal way in products shipped to customers in business intent on making money. Not in one-shot attempts, pilot projects or by non-profit organisations.
  A search on google skolar of industrial+software+formal+methods list papers from the 90s as the top results. And these seem more to be arguments against what I claim above rather than actual reports of usage. I really would welcome examples.

Software product line engineering

Software product line engineering is apparently about modelling variants and achieving formalism in feature descriptions. This is the conclusion I make after attending some workshops and the first morning sessions at the 14th International Software Product Line Conference in Jeju, Korea.
I presented an interesting  paper together with Håkan from Scania of how architects work with maintaining and updating existing architectures over time in the automotive industry. And we did not get a single question!
Last year when I presented another case study everybody was interested in the case and wanted to hear more from industry, but this time it didn't seem to interest the audience.
I find it quite difficult to find venues to present research based on industrial experience and not theoretical examples.

Besides that, I find the notion of feature used in many presentations different from what I am used to. To me a feature is something which is discernible for the end user, same as the definition found in the original work here. For example an adaptive cruise control is a marketable feature in a vehicle. But if I would model that similar to feature modelling prevailing here the model would consist of an optional radar, a compulsory engine, compulsory brake, etc.This means a much higher degree of knowledge about the realisation of features.

Tacit knowledge

At the ECSA conference there was a lot of talk about tacit knowledge and the importance of tacit knowledge when architecting systems.
I completely agree that one thing that differs an experienced (and productive) architect from a fresh graduate is is the amount of tacit knowledge the former possess. However I thought that in the discussions at the conference there was some confusion of what is exactly meant by tacit knowledge.
On one hand there was the view that tacit knowledge was simply the architectural knowledge that was not documented, i.e. stuff that only resided in the peoples' heads. To improve the management of this is mostly a question of capturing it in the right form and with the best tools.
On the other hand one sees tacit knowledge is such knowledge which is difficult to express using language, which is the tradition I'm used to.
In order to capture the latter type of tacit knowledge it is not just a question of having the time to do so or the right tools. The difficult part is for the person having this knowledge to be able to formulate in in such a manner that he can express it.