Langlebigkeit und Wartbarkeit

 

In diesem Post möchte ich das alte Dilemma der steigenden Wartungskosten von Software ansprechen - und auf ein dazu passendes Buch (von Carola Lilienthal) hinweisen. Ich habe ihn für dieses Buch als Geleitwort geschrieben (bin damit sozusagen parteiisch).

Eigentlich...

... wissen Softwareentwickler(innen) und –architekt(innen) ganz genau, worauf sie bei Entwicklung und Änderung von Software achten sollten: Einsatz etablierter Architektur- und Entwurfsmuster, saubere Modularisierung, lose Kopplung, hohe Kohäsion und Durchgängigkeit (Konsistenz und innere Ordnung), dazu eine große Portion sinnvoller weiterer Entwurfsprinzipien. Haben wir alle gelernt, geübt und erfolgreich genutzt.

 

Dennoch...

... geht in den Tücken der Praxis so einiges schief: Viele Softwaresysteme erkranken über kurz oder lang an der IT-Seuche Nr. 1 – der „generellen Verrottung“: Folgen dieser Malaise: 

• Wartungs- und Änderungskosten steigen unaufhaltsam auf ein schier unerträgliches Maß an. 
• Intransparenz wohin man nur schaut. Kaum jemand überblickt noch die Konsequenzen von Änderungen. Selbst kleine Erweiterungen werden zum technischen Vabanquespiel.
• Arbeit an solchen Systemen wird zur Qual – obwohl Softwareentwicklung an sich zu den interessantesten Tätigkeiten überhaupt gehört. ☹

Der folgenden Abbildung liegt kein reales System zugrunde, spiegelt aber meine Erfahrung in Dutzenden mittlerer und großer Systeme aus unterschiedlichen Branchen und Technologien wider:

 

 

Endlich...

... bricht Carola Lilienthal mal explizit die Lanze für die beiden (in der Praxis allzu oft vergessenen) Qualitätsmerkmale Langlebigkeit und Wartbarkeit. Diese beiden bilden eine wesentliche Grundlage der inneren Qualität von Software. Niemand kann sie einer Software von außen ansehen – aber alle Stakeholder von Systemen möchten sie haben:

• Auftraggeber von Systemen freuen sich, weil sich die hohen Investitionen in Erstellung und Wartung lohnen und weitere Änderungen kostengünstig sind.
• Anwender und Fachabteilungen freuen sich, weil Änderungen am System schnell und mit hoher Zuverlässigkeit erfolgen können. 
• Entwickler und Softwarearchitekten freuen sich, weil Arbeit an sauberen Systemen viel produktiver ist. Niemand braucht mehr Angst vor bösen Nebenwirkungen einer kleinen Änderung zu haben.

 

Prima

... dass Carola dieses Buch geschrieben hat: Ihre langjährigen Erfahrungen auf dem Gebiet der Architekturanalyse von Systemen unterschiedlicher Technologien sind einzigartig. Dadurch stellt sie in jedem Winkel dieses Buches den nötigen Praxisbezug her.Ich durfte als einer der Ersten das Manuskript lesen – und habe sehr davon profitiert!

 

Mehr dazu im Buch selbst (die Website zum Buch ist noch in Arbeit…)

 

History:

Nov.07: fixed typos (thx, @stilkov)

Software Quality is More Than Just Code Quality

I'm astonished how many people in our software business try to improve systems by JUST looking at source code - and ignoring loads of other aspects. Especially vendors of code analysis tools tend to equate "software quality" with "code quality" - which ignores major parts of quality issues. Let me illustrate my point with some examples.

Sample 1: Gold-Plated System

The team has created features above features, all highly configurable with a good-looking user interface. There's a button here, a menu there, some choices and a lot of functions... all stuff adding no business value, not useful for the end-user. By the way - all written in clean-code. In my opinion, this team produced *waste* - and the resulting system will more expensive to maintain than it needs to be - more lines of code, more configuration, more difficult to understand.

Sample 2: Overly Complex Solution

Data from a relational database is selected, then transformed into Java objects, then into an XML representation. This XML is processed into a pdf document, which is send via smtp-mail to the user. User prints the pdf, fills out some fields, scans the pdf. The system performs OCR (optical character recognition) and transforms back to xml. The xml is converted to Java objects, which are used to update the relational database. Sounds crazy? Is crazy - but describes an existing system. All written in nice and clean code. In my opinion this sample describes a conceptual disaster, where refactoring code alone wouldn't help - re-architecting the overall process would be the right approach.

Sample 3: Horrible User Experience

A company opened their impressive and useful backend services to customers by adding a web-frontend. Both back- and frontend code was written by expert programmers, adhering to many clean-code principles. Unit and integration tests supported the system. But the user interface has a nightmare. Layout, colors and screen flow had been created by programmers, not by usability-experts. Sidenote: I'm quite sure you have experienced similar situations... programmers-as-designers seems to be a common anti-pattern in development.

Software Quality is MORE than Code Quality

The examples above hopefully illustrate the point that even with well-written code a system can *fail* for its users or developers. Quality attributes like performance, understandability, maintainability, understandability, security or compliance might be neglected despite good source code...

Analyze Problems From Bird's Eye Perspective

In exiting business systems you should approach *improvement* or system evolution from different perspectives, not only by counting code-violations or code-metrics. For example the following perspectives provided valuable input for some of my own practice experiences:

• Qualitative Analysis (e.g. ATAM)
• Runtime Analysis, e.g. performance and resource monitoring
• Analysis of Development Process
• Data Analysis - find improper structures and content
• Issue-Tracker Analysis
• Static Code Analysis

Clean Code is Important...

I'm convinced of the importance of understandable code. I will actively promote refactoring towards cleaner code whenever appropriate. The principles of clean code need to be taught and practiced throughout our industry. But hopefully you see my argument - there's some more to do: we need to create understandability on a higher level, need to design appropriate technical and business concepts, need to design appropriate data structures and interfaces and so on...

Methodical Improvement to the Rescue

What's needed to systematically improve software is an iterative approach to find *many* problems, not only within code. aim42 collects practices and patterns to achieve exactly that - the foundation for systematic improvement. Think about it when some manager demands compliance to some code metric or coding-convention... you might be better off by optimizing or improving something completely different.

aim42 - Systematic Approach to Software Maintenance

During the OOP-Conference 2014, Stefan Tilkov and myself released aim42 - the systematic approach to software maintenance and improvement.

We named it "aim", short for "Architecture Improvement Method". aim42 is a free and open-source project, a community approach to collect practices and patterns.

Why Do We Need Software Architecture Improvement

Software maintenance is a major issue, because over time many systems degrade massively: Understandability (and with it, maintainability) goes down, and cost-of-change grow. See the diagram below:

aim4 Phases

It consists of three distinct phases:

1. Analyze: search and collect problems, issues, deficiencies and technical debt within your system and your development process.
2. Evaluate:understand root-causes of problems, determine their „value“ and cost/effort of their remedies.
3. Improve: systematically improve code and structures, reduce technical debt, remove waste and optimize.

Community Approach

Currently we host the (AsciiDoc) source of the aim42 guidebook on Github, one of the best-known open-source communities.
https://github.com/aim42/aim42
Find some further information (oops - currently in German only) within our initial presentation (on Speakerdeck) or the upcoming YouTube channel.

Changes

* Feb. 16th: Changed URL from Bitbucket to Github

arc42 now has open source repositories

At Bitbucket you now find our arc42.org open source code repositories. They are independent of the arc42 architecture template (will someday maybe serve as examples for the latter).
I just comitted two small utilities for Pdf manipulation (PdfStamper and PdfUtil) - both are not "finished" but working...

Issue trackers are live and actively monitored.

The third project, DupeDetect, handles duplicate entries in your geocaching logs… In case you don't know what that is, don't mind. Geocachers *will* know. Hopefully we make this available as a cloud-based service soon, parallel to the source code.
(did I ever mention that I absolutely *love* the Atlassian guys…)

Parallels, Windows, Enterprise-Architect and Retina

in principle, a great combo. With Win-7, Parallels works like a charm, nicely integrated with my Mac, no issues with the high Retina display resolution.

When I used EnterpriseArchitect for some UML modeling, I encountered serious display
issues: diagrams were suddenly scattered with overly large fonts and became utterly unreadable.

So back to virtual machine configuration: Shutdown the VM instance and open the Parallels configuration "graphics" tab. Choose "optimized for retina" mode, as shown below:

Not that difficult... boot Windows again.

Open the "display preferences" page and try the following screen resolution:

That completely solved the issue for me.

Find Stencils in OmniGraffle!

Did you note the cool new stencil-search in OmniGraffle 5:

Instead of restricting the search to the locally installed stencils, you can now search Graffletopia, the online stencil-collection.

Imho another proof that the OmniGuys are more than mere mortals!

Thanx, OmniGroup!

Neue Kolumne im Javamagazin: Knigge....

Gemeinsam mit Peter Hruschka veröffentliche ich im Javamagazin
eine Kolumne über Verhaltensmuster von Softwarearchitekten.

Ihr Titel lautet "Knigge für Softwarearchitekten - Verhaltensmuster zwischen Vorbild und Desaster"

Netterweise steht die auf jaxenter online!

Die Folgen:

Folge 1a) Verhaltensspektrum zwischen Sonne und Finsternis

Folge 1b) Der Proaktive

Folge 2) Der Elfenbeinturm

Folge 3) Der Vielsehende

Folge 4) Strukturierte Faulheit

Folge 5) Der Diktator

Folge 6) Blick in den Rückspiegel

Folge 7) Anti-Muster: Zuviel des Guten

Folge 8) Der Multilinguist

Folge 9) Der Notationskrieger

Folge 10) Der Code-Held

Folge 11) Die Jongleuse

Folge 12) Der Vereinfachungskobold

Folge 13) Der Perfektionist

14) Der technische Risikomanager

Und damit wäre DIESE Staffel erstmal beendet. Ab Oktober 2011 geht's weiter - also lesen Sie fleissig das JavaMagazin, denn dort werden die weiteren Folgen zuerst veröffentlicht.

Glaskugel: Architektur in 2020

Die Zeitschrift dotnetpro hat mich zur Zukunft von Software-Architekten befragt - und das Ergebnis im Rahmen eines gemeinsamen Glaskugel Artikels veröffentlicht (in dem auch noch Stefan Edlich und Michael Wiedeking ihre Meinungen zu Non-SQL beziehungsweise Programmiersprachen darlegen).

In Kurzform meine Thesen für die Zukunft von Software-Architekten:

1. Standardisierung von Aufgaben und Ausbildung: Hier gibt es vielversprechende Ansätze (beispielsweise iSAQB und der sehr enterprise-lastige TOGAF).


2. Anforderungsanalyse wird von Software-Architekten wahrgenommen. Im Bauwesen und großen Teilen der Ingenieurdisziplinen gibt es kein Requirements Engineering, sondern Konstrukteure. Das wird bei Software zukünftig ebenfalls so sein!


3. Projektmanager werden zu organisatorischen Erfüllungsgehilfen von Software-Architekten. Wer kennt schon den Projektleiter des genialen Kuppelbaus auf dem Berliner Reichstages? Niemand. Die meisten haben von Sir Norman Foster, dem Architekten, aber schon gehört!


4. Nichtfunktionale Merkmale als wesentliches Erfolgskriterium. Von der Einstellung Hauptsache es funktioniert... müssen wir uns ganz schnell verabschieden. Es geht um Qualität und Langlebigkeit!


5. Um Größenordnungen verbesserte und komplexere Werkzeuge. Nur noch wenige Eingeweihte werden die Interna zukünftiger Entwicklungswerkzeuge verstehen. Sie werden ungeheuer viele Details vor uns Normalbürgern wegabstrahieren - einen Vorgeschmack gibt das für Kinder entwickelte Programmiertool Scratch - das beispielsweise das komplexe Thema Threading verbirgt - und zwar perfekt!
   

Als Software-Architekten haben wir eine wirklich spannende Zukunft vor uns!

ANTLR plugin on IntelliJ abandoned

too bad - one more plugin that doesn't work properly with my beloved IntelliJ: ANTLR works grammar development.

I inquired with Terence Parr, the definitive source for all ANTLR infos - and he admitted that the ANTLR plugin for IntelliJ has been abandoned, as it was "too difficult to get it to run properly".

Rescue is on its way - ANTLRv3IDE for Eclipse (3.5 onwards). Great plugin, great docu.

I stopped using Eclipse...

Ever had hassle with your numerous Eclipse plugins? With update-manager complaining about missing dependencies, which nobody ever could resolve?

With re-installing your (free) development environment every now and then?

After giving the latest Eclipse ("Galileo") a try, I just dumped it: After configuring Spring, Scala, webservice, subversion and a few other plugins, the whole app did not even bother to start...

Having been Eclipse user for several years, I am really grateful for their contribution to (Java) software development. But it has become an overly complex monster in my (humble) opinion.

I lighthearted moved over to IntelliJ-IDEA - their slogan "develop with pleasure" became true within the first hour of using IDEA (I tried both 8.1.3 and their early-access version 9). Grails and Groovy support is excellent, Scala plugin working fine, subversion support the best I ever had, even Clojure working (ok - that one can definitely be improved, but true lisp programmers should stick to Emacs)

Proud user of

Cool English-German Translation on Mac

Just in case you're offline sometimes and cannot use the fantastic dict.leo.org service:
Philipp Brauner made the dict.cc project available for Mac-OS (Leopard) with a really cool plugin.

It's free and well integrated into the standard dictionay application.

(You Mac users did know that you can translate any (highlighted) word from any Cocoa application by hitting Ctrl-Cmd-D, did you?)

VNC-Clients for Mac-OS

(Just in case you don't know VNC, skip this post...)

For those interested in using VNC to remote-control another computer... a simple solution
without the need to configure X-Servers...

At least two (free) options for the Mac are available:

• ChickenOfTheVNC: Nice, but remember to add the port-number in the address-field (e.g. 192.160.0.17:5901). What shall I say - it just works.


• JollyFastVNC: Faster and more responsive then the "Chicken", but does not properly map the (German) keyboard... Therefore, for me as DE-localized user it is close to unusable (except I only use the mouse).

Sh** happens - therefore everybody needs a backup strategy

Did you ever lost data due to an accidental "rm -r *" command? Or because your spouse just installad the newest service pack of any operating system which caused your machine to cease working?

I know about failures of humans and machines. I never want to loose digital assets due to such failures, therefore I designed and implemented a broad backup strategy.

But first things first. Let's start with the kinds of sh** that can happen:

Risks to Digital Assets

• Theft: Some nice guy steals your precious notebook with your even more precious digital assets on its drive.


• Damage: Your drive gets sick due to headcrash or an overdose of spilled coffee (I mostly stick to green tea, but they say that is equally fatal for harddrives).


• Accidental deletion: Wether it is "rm -r *" on unix machines or its equivalent on graphical user interfaces - users sometimes (involuntarily) erase their own data. All right, you might get it back from the recycle bin... unless somebody cleans that one


• Viruses or other data corruption: My special friends are office applications, killing large documents. I'm sure you've experienced that one yourself...

My Goals Concerning Backups

• Continue working as fast as possible.


• Highly automated backup - the fewer manual processes the better.


• Cover different computers with various kinds of valueable data


• Affordable. Keep the price-tag in personal range.

My Infrastructure

I'm working as freelancing IT-consultant in Germany, therefore I don't have too many machines to care for. The following diagram shows my infrastructure - which is primarily used by my wife (Cheffe Uli, for text processing, billing, accounting and other organizational tasks required to run a business).

• My primary machine is a MacBook Pro


• I use two different external USB drives. They contain all our digital music (approx 40GByte) and a copy of all our photos.


• An old PC runs OpenSUSE Linux 10.2 (very smoothly!) and contains two 160 GByte drives, which I configured as RAID-1 (mirrored) with the SUSE Yast configuration utility. The OS itself is located on a third drive. A few directories of the RAID can be mounted via Samba on our other machines.

The Pillars of My Backup Strategy

1. Of course I use a version control system (Subversion). I setup my (home) repository on our win-XP machine. Every project file I work on is regularly commited to subversion.


2. I fall in love with CrashPlan, a Java-based (commercial) lifesaver to backup arbitrary files and directories to arbitrary other machines (which must have a free version of CrashPlan installed). I backup the following things:
   
   
   
   • My Mac's valueable data to our home PC and to my RAID-1 linux server.
   
   
   • My wife's data from the home-PC to my Mac and to the RAID-1 server.
   
   
   • Not on the diagram: I backup all our data to my parents PC.
   
   
   
   


3. Every week I copy my complete MacBook-drive to one partition of an external drive (with SuperDuper). Uneven weeks are copied to partition #1, even weeks go to partition #2.


4. All my valuable data (especially project-related files and photos) are copied to Amazon S3 (TM) offsite storage with a small utility called JungleDisk (a little webdav-server). JungleDisk encrypts the files, so nobody without my key can read my files.


5. I copy the whole disk of our home-PC with Acronis TrueImage.


6. (not on the diagram): I keep a USB-stick with a pre-configured JungleDisk installation with me at all times. In case I need to access some files, I just plugin the stick into ANY machine (Win, Mac or Linux) with an Internet connection - and can start working a few minutes later...


7. (not on the diagram): I use Versomatic on the Mac (there is a Windows equivalent called RealTime Backup), which stores versions of all my working files in its own repository. Whenever I save a file in any application, Versomatic creates a new version in its repo... so I can go back in time (like TimeMachine, but I can configure Versomatic to just backup what I need!).


8. (not on the diagram): I backup files with iBackup - creating timestamped backup-sets on external drives or sometimes even on CDs or DVDs. I tried several others, including DobryBackup, but iBackup is simply better...


9. (not on the diagram): I keep a copy of SpinRite, the awesome disk analysis and repair utility from fabulous Steve Gibson. It can recover even bad disk failures (but is way to slow for USB-connected drives...)

Now what?

Let us review the list of risks from the beginning:

• Theft: If somebody steals my Mac, I'll have to get a new one. Bad enough. I'll be productive within minutes due to my SuperDuper backups.


• Damage: Every drive in our infrastructure is backed up somewhere. It might take a while to replace the computer, but I can easily restore every file, from JungleDisk or CrashPlan or Subversion or another of my copies.


• Accidental deletion: It'l be fatal if I deleted my subversion repository, but even that is safely copied by CrashPlan to several destinations (and not automatically deleted there!). Looks like I can survive a few of my own brain-blackouts...


• Viruses or other data corruption: See above. Several possibilities to get back to consistent versions, depending on the kind of damage.

I'm not fully automated in my backups - but that's ok for me.


Apart from those (technical) risks, what else can happen?

• You forget your passwords? Print out a list of important passwords, seal it in an envelope and deposit with trusted friends.


• Your house burns down. All computers within are destroyed. As long as you remember your JungleDisk password or your CrashPlan-ID, you're done... External backups have their merits...


• I keep a number of boot-CD's ready... for example a grand-universal-windows-boot-disk

Conclusion

I'm pretty sure you guys come up with some risks I forgot. The major pillars in my backup strategy are CrashPlan, JungleDisk and redundancy...

OOP 2008: Architektur braucht Iterationen

James Dyson, Erfinder des nach ihm benannten (ungeheuer lukrativen, coolen und innovativen) "beutelfreien" Staubsaugers, hat 15 Jahre lang iteriert, seine Architektur an 5000 Prototypen getestet (so berichtet die Atlantic Systems Guild in ihrem genialen Buch "Adrenalin-Junkies und Formular-Zombies" (Pattern 84, Schonzeit für Ideen).

Sagt am Beispiel eigentlich alles. Hervorragende Architekturen brauchen ein paar Iterationen (manche mehr, manche weniger). Während dieser Iterationen ändern sich einige Anforderungen - und schon haben wir die Agilität im Spiel.

Also: Sieht in etwa wie folgt aus: Kunde und Umwelt ändern Anforderungen, Architekten reagieren...




Ein aus meiner Sicht halbwegs realistischer Prozess der Architekturentwicklung sieht etwa so aus:



Sie sehen: Schleifen. Iterationen. Versuch-Scheitern-Lernen-Verbessern. So funktioniert Architektur.

Lisp again: Eclipse plugin

Great news:

InfoQ reported about it... Cusp, an Eclipse plugin for Lisp.

One day, when I've spare time, I'm gonna post an experience report :-)
(so you might be better off NOT to wait for it...)

A free, nice (and complete) book on Linux (in German)

Galileo Computing, a German publisher, made its impressive Linux Handbook
(written by J. Plötner and S. Wendzel) available for download...

more than 1100 pages, covering all Linux distributions. A nice reference, well written.

(its also available in print, with two DVS's, loads of video-tutorial stuff etc.)

Maaaany bugs fixed in Drools...

The Drools-team just published their maintenance release 4.0.1 with loads of bugfixes (described here).

Thanx for your good support, guys :-)

Intelligent backup solution

At least I found a backup solution which suits my needs: CrashPlan can either backup to a central server, but (really cool!) also to your other machines... (Mac, PC, Linux!!)

I had it up and running in a few minutes - tried their support (reactive, competent) and had 50GBytes backed up from my Mac to our family-PC (yes, we're still using it...) within a few hours...

Elegant UT - a cool addition to my backup strategy... (20-50 Euro).




Update (Sept 07): I verified the following things to make sure it works as expected:

• Simple restore of a file or a previous version. Crashplan can keep arbitrary versions :-)


• Restore from a different machine, so I can restore files even if the original machine is stolen, burnt down, drowned or otherwise unavailable.


• Asking support questions: The CrashPlan support guys are responsive, friendly and competent. Excellent marks here!

Golfer-Riddle (Prolog-Version)

In my ongoing quest for improved awareness for rule-based systems, I solved the golfer-riddle in Prolog and published the solution together with an explanation here.

It's a pretty straightforward translation of the riddle itself - Prolog rocks!
(I'm open to improvisation suggestions...)

gcolor(blue).
gcolor(plaid).
gcolor(orange).
gcolor(red).

gpos(1).
gpos(2).
gpos(3).
gpos(4).

golfer( [_, C, P] ) :- gcolor(C), gpos(P).
		
golferRiddle( G1, G2, G3, G4) :-  golfer(G1),golfer(G2),golfer(G3), golfer( G4),
	G1 = [fred, FredCol, FredPos ], 
	G2 = [joe, JoeCol, 2],
	G3 = [tom, TomCol, TomPos], 
	TomPos =\= 1, TomPos =\= 4,
	TomCol \= orange,
	G4 = [bob, plaid, BobPos],
	
	/* unique colors */
	is_set([FredCol, JoeCol, TomCol, plaid]),
	
	/* unique positions */ 
	is_set([FredPos, TomPos, BobPos, 2]), 
	 
	/* Fred's right neighbour wears blue */
        plus( FredPos, 1, FP1),
        member( [_, blue, FP1], [G1, G2, G3, G4]).

Pragmatic Re-Re-Branding

A good move: The (awesome) rule-engine "Drools" is back- including a new version of its logo:



Mark Proctor writes in his blog:

"Having received community feedback 4.0 will start the push to reclaiming the Drools name.... So to help speed up this process, I would please encourage everyone now to use "JBoss Drools" in any articles, blogs, talks etc."

I'll start: JBoss-Rules will be called "JBoss-Drools" from now on (again)...