The Four Steps of Complex Tasks (Part II)

In the first part of this blog entry, we talked about how complex tasks need to be addressed with a proper problem-solving framework. One such mental framework can be found in traditional warfare. It involves all the anticipated artifacts like headquarters, mission statements and a general’s map, but will likely omit the gruesome parts of actual battle.

We started with the mission statement and then began to make a plan with four steps:

  • Reconnaissance
  • Maneuver
  • Offensive
  • Defensive

Step one: Reconnaissance

In the first step, we tried to unveil every part of the scenario and draw a complete map of the terrain. A botched reconnaissance is probably the root cause of most failed missions. You can read all the details about the reconnaissance step in the first part of this blog entry.

Step two: Maneuver

Emergency preparation equipment on the grass, on the nature backgroundFor a real army, maneuvering means to “change position”. In preparation of a battle, it means to secure the positions that will maximize the own effect and/or minimize the effect of the opponent. Most battles are already decided in this phase, with the following fighting being more or less the playback of the drama the generals anticipated. The ultimate victory in military warfare is the victory by maneuver, when the opponent revises his position before the battle and concedes that he lost already.

In our example case, we wage war on the call for proposals for a big software project. It would be our ultimate victory if we could convince the project owner that no call for proposals is even necessary because we are clearly the best-fitting proposer. But that would have required actions from our side in the past and that chance has passed. We need to prepare for the “fight” under the rules of the project owner, we need to submit a better proposal than everybody else.

Our maneuver step contains every preparative action we need to do so we can play out the last two steps in a smooth fashion. If we need to create an account to submit our proposal, then now is the time to create it. If we need to buy some office supplies to print the proposal in top-notch quality, we should buy them now. Just like a real army stocks their supplies near the anticipated battlefield, we need to stock our supplies, physical like the office supplies or virtual like the user account or a signing certificate.

The goal of maneuvering is to never stall when the last two steps are due. We take our knowledge from the reconnaissance step and interpolate it into the future. The maneuver actions support our scenario of the future. Once the third step is in progress, every negligence in maneuvering will mean delay, makeshift solutions and partial failure. If the negligence is too widespread, it will result in overall failure.

Step three: Offensive

Hiker crossing rocky terrain in the Bryce Canyon National Park, USAIn a real battle, once the maneuvering is done, things “get real”. This usually means that shots are fired. In our example, we also fire shots, but imaginative ones. During the offensive, we really work on the meat of the proposal. We dig into the details of the project and produce estimates and concepts. We use the mandated structure for the proposal to fill our proposition in. We concentrate on fabricating content.

In this phase, things get messy and confusing very fast. There are just way too many details needing attention all at once. This is where our plan from the reconnaissance step comes to our rescue. We need to make sure that we don’t stray from the plan too much. Remember, our “opponent” isn’t moving, it’s a static target. So our plan will stay mostly valid during the offensive. If not, this indicates flaws in previous steps and should be taken seriously. If you can afford it, time- or effortwise, rewind your mission back to step one if you find yourself attacking dummy problems or empty terrain that leads you nowhere. A well-planned offensive has immediate and visible effects.

Your work during the offensive phase might look chaotic and erratic from the outside, but it should be cold-blooded and calculated in your experience. This phase is known to intimidate you with overwhelming feelings of anxiety and despair. Stick to your plan and don’t panic! If you’ve planned it well, it will go well. If you didn’t trust your plans beforehands, why would you even proceed to this step? There is no damage done when your reconaissance unveils a task to heavy for your taste and you make an immediate retreat. There is little loss in surrendering your efforts to an opponent that played the maneuvering game better than you, like requiring several comparable projects as reference for the proposal, but you are a newcomer on the market. It will ache, but you cut your losses and move on. But starting an all-out offensive that you are not sure you’ll win? That’s just stupid or desperate.

Two remarks here: First, Being sure you’ll win means you are sure to fulfill your mission, in our example to submit a valid proposal. That doesn’t imply you need to be sure to win the pitch itself. Stick to your mission statement and win the battle before you try to win the whole war. Second, if you hold back on your offensive, you set yourself up for failure because of indecision and foot-dragging. Every offensive should be all-out or not started at all. You are in this game to win, not to play.

Step four: Defensive

Let’s assume our offensive was successful. In a real battle, we have conquered the enemy’s stronghold or additional terrain. The enemy is defeated. A movie would now show the end credits, a computer game the game results. But this is real life, there is no “end point”. Your troops are all over the place, probably in a sorry shape and without a clear goal to look forward to. If your enemy has any troops left, now is the best moment to run you over. Your victory would be pyrrhic, your winning would finally cause your defeat.

The clever strategist has already planned the defense after the offensive (and victory). Often, this means a partial retreat after the battle in order to “straighten the lines”. We can’t do that in our example, it would mean we take back promises after our proposal wins the pitch. But we can plan our defense after victory.

Let’s assume our proposal wins. What does that mean for our company? Who will work on the project? Can we keep our promises even if external circumstances like other projects, other proposals or our staff changes? What will we gain from the project? What concessions can we make to the customer if he wants to re-negociate? Do we need to re-negociate as soon as the agreement is made? The last question answered with yes is a typical sign of over-commitment in the offensive phase and tells about poor leadership.

In our example, let’s say we’ve promised the customer a 24/7 support hotline for the software. We need to make sure how to fulfill this promise before we send out our proposal. There is no value in making hollow promises that we cannot keep. This would be like losing captured terrain again just because you cannot provide enough troops to secure it. It’s not worth the effort and an all-around damper on morale. Just to be clear here: You don’t need to act on the fullfilling of the promises before you’ve won, but you cannot wait with the planning. So we need to have a clear plan on how to implement a 24/7 support hotline, but we only need to act on it as soon as we are sure it is really necessary. We need to take steps one and two for the support hotline mission, but hold back the later steps until our proposal has won and the contracts are signed.

Don’t omit this step in your planning. A successful offensive without the backing of a good defensive is the prelude to a disaster.

Conclusion

We’ve learnt the four steps to master each complex task, lent from the art of warfare, namely reconaissance, maneuver, offensive and defensive, that form a pattern you can repeat each time with the same structure, but always different content. Every task will require a different solution, but the solution’s framework is always the same. This framework can be applied to tasks that seem to have nothing in common with warfare, but still play by the same rules. This is a powerful tool because it opens centuries of knowledge in military warfare to your creative transfer approach. And it is an effective tool because you don’t need to study history to apply it to your cause. Just reiterate the four steps and conquer your task.

If you’ve already applied these four steps, perhaps without consciously realizing it, I would love to hear your story and the outcome. Please leave your comment below!

The Four Steps of Complex Tasks (Part I)

In software development, we often face complex tasks that need to be addressed with nearly certain success rates. “Failure is not an option” is an easy sentence to say, but hard to implement. As experienced software developers, we’ve learnt our share from failures or near-misses and have probably derived an implicit problem-solving framework for our typical tasks. But what if the tasks are not so typical? What if we need to do complex work where we lack experience – and know it?

Adopting a problem-solving framework

In order to keep this blog entry as practical as possible, lets tell the theory with the help of an example that will be foreign to most software developers: You need to participate in a call for proposals for a big software project. You are responsible for the complete process from downloading and reading all documents for the call for proposals, making sense of the requirements, making a plan and an estimate, writing everything up and delivering your proposal in time and form at the right address. You cannot rely on the help of more experienced colleagues. You will probably feel overwhelmed and a bit helpless.

Now is the time to look for proven problem-solving frameworks that will at least help you coordinate your work and remind you about necessary steps that might go by the board otherwise. There are probably numerous frameworks out there, but I’ve found one to be easily applicable and reliable in the outcome. It’s the four steps of warfare. “Warfare” is a heavy word that isn’t in the dictionary of most people. It really means “to wage war” on somebody or something. But it doesn’t imply death, injury or even a “loser”. If you can wrap your head around the following sentence, we can easily forget its ancestry and follow the steps without any bloodshed: We are about to wage war on the call for proposals.

Being on a mission

We are now on a mission! Every mission starts with the most important piece of information: The mission statement. In this statement, the primary goals of the mission need to be declared as clear and unambiguous as possible. In our example, the mission statement might read like “I want to deliver a complete, believable and feasible proposal for the software project while adhering to all formalities”. It is worthwhile to write the mission statement down and keep it visible. This is your goal, this is why you wage war. Anything you do during the mission that doesn’t help you further your case in regard to the mission statement can be called a distraction (or in agile speak: “waste”).

Analyzing our example’s mission statement, we can conclude that we have two fields of work to cover: the content of the proposal and the formalities of the call. Both need to be covered successfully in order to achieve mission success. It is not enough to have a formidable proposal but send it in too late or without certain required characteristics. And it doesn’t do good to send something awful in that fulfills all technical requirements of the proposal but stands no chance because of its content. You’ve just discovered that your war will be fought on two fronts (content and formalities) and neglecting either one of them will not result in victory. That is an important discovery for the moment you employ your “troops” – time, attention, effort and material in our case.

The four steps

But slow down! We are far away from employing our troops! We first need to learn about the four steps of warfare. With our mission statement at hand, we have to make a plan for these steps:

  • Reconnaissance
  • Maneuver
  • Offensive
  • Defensive

Nearly every military mission can be broken down into these four steps in that order. We cover the first step in this blog post and the following three in the second part next week:

Step one: Reconnaissance

hunter looking through binocularsEvery warfare mission has important things at stake. There would be no mission otherwise, because each mission is expensive. With something big at stake, you want to reduce risk as much as possible. The military way to reduce risk is to gather knowledge about the enemy, to “increase intelligence”. Our “enemy” is the call for proposals with its deadline and rules and requirements. It is an easy enemy in a way, because all information is already made available in clear form and will not change over time. Or is it? Our first move should be to gather and revise all documents that tell us something about the call for proposals. Are we in possession of all available information? Nothing will break our neck more effectively than forgetting to read all documents. The sentence “I didn’t know that!” might sound like a good explanation why the mission failed, but it only indicates that you already failed in the reconnaissance step. Everything you do after a botched reconnaissance has a high risk of being ineffective, useless or downright counter-productive. You need to be sure to survey the complete terrain (read all the documents). A general that says “we don’t need to look behind this mountain, there will be nothing behind it” is in danger of failure as sure as you are if you say “that’s probably all information there is, what else should one want from me?”. Don’t underestimate your enemy! Just because you can download and read everything in regard to your call of proposals doesn’t mean it can be found at one place or will be conveniently available. Double-check that you’ve followed each hint about additional rules (like in general terms and conditions). Read all documents carefully!

When you’ve gathered a complete map of the terrain (have all rules and requirements in readable form), it’s time to mark the problematic features. Make a list of all formalities you need to adhere to. This list is already part of your battle plan. You will later need to employ resources to check off each point of this list. Mark all requirements that you find problematic or surprising. Those are like the strongholds on a battlefield. They need extra attention from you later on. Take your list of simple tasks and tack it on your mission board. If you didn’t have one yet, you’ll have it now. The mission board will look exactly like in the movies when you complete your first step. On top goes the mission statement, with your task list in whatever fashion suits you beneath it (some prefer the ordered list, some might draw a graph, whatever works for you). The mission board is for you and you only, nobody else needs to understand it. You’ve just created your headquarter! Now you have all ingredients to assemble your troops on the battlefield: you know the terrain, where the enemy resides, where you are and what your mission is.

Now is the time to get moving – in part two of this blog entry series next week. Stay tuned!

A simple yet useful project metric

In my years of managing software development projects, I’ve come to apply a simple metric to each project to determine its “personality”. The metric consists of only two aspects (or dimensions): success and noise. Each project strives to be successful in its own terms and each project produces a certain amount of “noise” while doing so. Noise, in my definition, is necessary communication above the minimum. A perfectly silent project isn’t really silent, there are just no communicated problems. That doesn’t mean there aren’t any problems! A project team can silently overcome numerous problems on their own and still be successful. The same team can cry for help at each and any hurdle and still fail in the long run. That would mean a lot of noise without effect. I call such a project a “Burning Ox”.

Success vs. Noise

metric

As you can see, there are four types of projects with this metric. The desired type of project is the “White Knight” in the top-left quarter, while the “Burning Ox” in the bottom-right is the exact opposite. Let’s review all four types:

  • White Knight (silently on track): A project that is on track, tackles every upcoming challenge on its own, reports its status but omits the details and turns out to be a success is the dream of every project manager. You can let the team find its own way, document their progression and work on the long-term goals for the team and the product. It’s like sailing in quiet waters on a sunny day. Nothing to worry about and a pleasant experience all around.
  • Drama Queen (loud, but on track): This project is ultimately headed towards success, but every obstacle along the way results in emergency meetings, telephone conferences or e-mail exchanges. The number of challenges alone indicate that the team isn’t up to the task. You are tempted to micro-manage the project, to intervene to solve the problems and ensure success or at least progress. But you are bound to recognize some or even most problems as non-existent. The key sentence to say or think is: “Strange, nobody else ever had this problem and we’ve done it a dozen times before”. If you are a manager for several projects, the Drama Queens in your portfolio will require the majority of your time and attention. You’ll be glad when the project is over and “peaceful” times lie ahead.
  • Backstabber (silent and a failure): This is the biggest fear of every manager. The project seems alright, the team doesn’t report any problems and everything looks good. But when the cards need to be put on the table, you end up with a weak combination. It’s too late to do anything about the situation, the project is a failure. And it failed because you as the manager didn’t dig deeper, because you let them fool you. No! If you look closer, it failed because nobody dug deeper and everybody was in denial. You’ll see the warning signs in retrospective. You will become more paranoid in your next project. You’ll lose faith in the project status reports of your teams. You’ll inquire more and micro-manage the communication. You’ll become a skittish manager because of this unpleasant experience. Backstabber projects have horrendous costs for the social structure of a company.
  • Burning Ox (loud and failing): The name stems from an ancient war tactics when the enemy’s camp was overrun by a horde of oxen with burning torches bound to the horns. The panicked animals wreaked havoc along their way and started fires left and right. A Burning Ox is helpless in the situation, but takes it out on anybody and anything near it, too. This project is bound to fail, the team is in it way over their heads and no amount of support from your side or help from the outside can safe it. Well, experienced firefighters might work wonders, but they are expensive and rare (we know because we are often called in for this job). If you find a Burning Ox in your project portfolio (and you will know it, because a Burning Ox screams on the top of his lungs), prepare yourself for the inevitable: The project will fail, in scope (missing functionality), budget (higher costs) and/or time (delayed delivery). You better start with damage control now or make a call to a firefighter you can trust.

Easy assessment

This project management metric is not meant for deep inspection, but for easy assessment and quick communication. You can convey your desired communication style and the fact that everybody involved with the project is partly responsible for its success or failure. The metric states that too much detail is not helpful and too little detail can be disastrous. It also shows that loudly failing projects are not the fault of the project team alone (the ox cannot help being used as a living torch), but that the prerequisites of the project weren’t met.

Takeaway

If you are not a project manager, what can you learn from this blog post? Ask yourself if you require too much help from your manager, forcing him/her to switch into the micro-management gear, even if you could solve the problem yourself. If you cannot, ask yourself if you think that you can deliver the project in scope, time and budget or if you already smell the fire. If you can smell the fire, is your manager aware? Are you telling him/her in unclouded words about your perceived state of the project? Did you attempt to communicate your perception/feeling at least twice? If not, your manager might be shocked that he/she took care of a Backstabber project. A failing project is not your fault! You would only be to blame for the continued hiding of a known fact.

If you are a project manager, take a piece of paper, draw the metric’s chart and try to pin-point the position of all your projects. Be as honest and exact as possible. Is it really a Burning Ox or “just” a Drama Queen? Are your White Knights really above reproach or is their loyality questionable? What questions could you ask to try to unveil hidden problems, even those that nobody is aware of yet?

These quick, repeated assessments help me to manage my schedule and not forget about the silent projects because the loud projects always ellbow their way into my attention.

Recap of the Schneide Dev Brunch 2016-12-11

brunch64-borderedLast week at sunday, we held another Schneide Dev Brunch, a regular brunch on the second sunday of every other (even) month, only that all attendees want to talk about software development and various other topics. This brunch was so well-attended that we had to cramp around our conference table and gather all chairs on the floor. As usual, the main theme was that if you bring a software-related topic along with your food, everyone has something to share. Because we were so many, we established a topic list and an agenda for the event. As usual, a lot of topics and chatter were exchanged. This recapitulation tries to highlight the main topics of the brunch, but cannot reiterate everything that was spoken. If you were there, you probably find this list inconclusive:

Finland

We started with a report of one of our attendees who had studied in Finland for the last two years. He visited the Aalto university and shared a lot of cultural details about Finland and the Finnish people with us.

The two most important aspects of the report were sauna and singing. The Finnish love to visit a sauna, in fact, nearly every building has a functioning sauna. Every office building has a company sauna that will get visited often. So it might happen that your first visit of a company starts right in the sauna, naked with the bosses.

And the Finnish love singing so much that they usually start singing during the sauna session. There are open social events organized around singing together.

Alcohol plays a big role in Finland, mostly because the taxes makes it incredibly expensive to obtain a proper buzz. In the southern regions, much alcohol is imported from Russia or Estonia by ferry. There are even special ferry routes designed to be cost-neutral when shopping for alcohol. But alcohol isn’t the only thing that is made expensive with special taxes. Sugar and sugary food/drinks are heavily taxed, too. So it’s actually more expensive to eat unhealthy, which sounds like a good concept to counter some civilizational diseases.

The Finnish students often wear a special boilersuit during official events that identifies their affilition with their field of study and university. They apply patches and stickers to their suit when they have completed certain tasks or chores. It’s actually a lot like a military uniform with rank and campaign insignia. Only that the Finnish student boilersuit may not be cleaned or washed other than jumping into a body of water with you in it. And the Finnish lakes are frozen most of the year, with temperatures of -27 °C being nothing extraordinary.

As you probably have guessed right now, costs for rent and electricity are high. Our attendee enjoyed his time there, but is also glad to have the singing separated from the alcohol for the most part.

Lambdas and Concurrency

The next question revolved around the correlation between lambda expressions and concurrent execution of source code. The Vert.x framework relies heavily on lambdas and provides reactive programming patterns for Java. As such, it is event driven and non blocking. That makes it hard to debug or to reason about the backstory if an effect occurs in production. The traditional tools like stacktraces don’t tell the story anymore.

We took a deep dive into the concepts behind Optionals, Promises and Futures (but forgot to talk about the Expected type in C++). There is a lot of foggy implementation details in the different programming languages around these concepts and it doesn’t help that the Java Optional tries to be more than the C++ Optional, but doesn’t muster up the courage to be a full Monad. Whether deprecating the get()-method will make things better is open for discussion.

To give a short answer to a long discussion: Lambdas facilitate concurrent programming, but don’t require or imply it.

React.js and Tests

It was only a small step from the reactive framework Vert.x to the React.js framework in Javascript. One attendee reported his experiences with using different types of tests with the React framework. He also described the origin of the framework, mentioning the concept of Flux and Redux along the way.

Sorry if I’m being vague, but each written sentence about Javascript frameworks seem to have a halflife time of about six weeks. My take on the Javascript world is to lean back, grab some popcorn and watch the carnival from the terrace, because while we’re stuck with it forever, it is tragically unfortunate. Even presumed simple things like writing a correct parser for JSON end in nightmares.

It should be noted, though, that the vue.js framework entered the “assess” stage of the Thoughtworks Techradar, while AngularJS (or just Angular, as it should be called now) is in the “hold” stage.

Code Analysis

We also talked about source code analysis tools and plugins for the IDE. The gist of it seems to be that the products of JetBrains (especially the IntelliJ IDEA IDE) have all the good things readily included, while there are standalone products or plugins for other IDEs.

Epilogue

As usual, the Dev Brunch contained a lot more chatter and talk than listed here. The number of attendees makes for an unique experience every time. We are looking forward to the next Dev Brunch at the Softwareschneiderei in February 2017. We even have some topics already on the agenda (like a report about first-hand experiences with the programming language Rust). And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Let’s talk about C++

It’s almost time for the holidays again. A time to reminisce. A time for family. A time for community.

Us software developers seem like an odd folk. We spend endless hours tinkering with our machines and gadgets. It appears like a lonely profession to outsiders. And it can be. Sometimes we have to get in The Zone to solve our tasks and problems. Other times we need to have sword fights. But sometimes we just have to meet other developers.

I’m not talking about your 10 o’clock daily standup or agile flavor-of-the-month meeting with other departments. Those are great. But sometimes it just has to be us programmers, as tech people.

Let’s talk about cool and tricky algorithms. Let’s talk about the latest and greatest language features that make all code some much cooler. Let’s talk which editor is the greatest. All the technical details.
It’s not necessarily the most important and essential aspect of our craft, no. But it’s kind of like the seasoning to a well cooked meal. It’s flavor and character. It’s fun.

I’m the C++ guy. It’s not the only one of my specialties, but kind of what I got a bit of a reputation for. And I like to talk about it. So far, this was either limited to colleagues and friends or “out there” on IRC, stackoverflow or other online communities. But I want to extend that and be a more active member of the local community.
David Farago had the great idea to create a platform for this in Karlsruhe: The C++ User Group Karlsruhe. He asked me to kindly extend an invitation. The kick-off is next month, right at the start of the new year, on the 11th of January, with one meeting scheduled every month. I think this is a perfect time to do this. C++ is in a great place right now. The language is evolving in a very positive way and the ecosystem is looking better and better.
So if you’re in any way interested meeting other local C++ people, please join us. I’m very much looking forward to meeting you guys!

Displaying numbers in tables

Many software applications have to display series of numbers, for example statistical information, measurement values or financial data. Of course there are many ways to visualize values graphically with charts, but sometimes the user wants to see the actual values as numbers. The typical layout method to display numbers are tables.

Here are some guidelines you should follow when you have to display numbers in a table.

Integer numbers

Right aligned integer numbers

Right-aligned integer numbers

Integer numbers that are shown in a table column should be right-aligned, because the orders of magnitude of a number’s digits increase from right to left. Additionally you should choose a font with fixed-width digits for numbers. This ensures that digits with the same orders of magnitude line up. Thus the numbers can be compared more easily. The font itself doesn’t have to be a fixed-width font in general. Some proportional fonts with variable widths for letters have fixed-widths for digits, called tabular figures.

Non-integer numbers

Aligned with decimal points

Aligned with decimal points

Non-integer numbers with decimal points should be aligned with their decimal points. The reason is the same as above: digits with the same orders of magnitude should line up. This can be a bit more effort to implement in your application than mere right-alignment, because components such as UI widgets or HTML tables usually don’t directly support this form of alignment.

However, you can implement it by using a font with tabular figures and then right-pad the numbers with spaces. Each of these spaces must have the same width as a digit, of course. This is the case with a fixed-width font, but there is also a special Unicode character for this purpose that can be used with proportional fonts and tabular figures: it’s called figure space and has the Unicode code point U+2007.

Modern developer Issue 4: My SQL toolbox

SQL is such a basic and useful language but the underlying thinking is non-intuitive when you come from imperative languages like Java, Ruby and similar.
SQL is centered around sets and operations on them. The straight forward solution might not be the best one.

Limit

Let’s say we need the maximum value in a certain set. Easy:

select max(value) from table

But what if we need the row with the maximum value? Just adding the other columns won’t work since aggregations only work with other aggregations and group bys. Joining with the same table may be straight forward but better is to not do any joins:

select * from (select * from table order by value desc) where rownum<=1

Group by and having

Even duplicate values can be found without joining:

select value from table group by value having count(*) > 1

Grouping is a powerful operation in SQL land:

select max(value), TO_CHAR(time, 'YYYY-MM') from table group by TO_CHAR(time, 'YYYY-MM')

Finding us the maximum value in each month.

Mapping with outer joins

SQL is also good for calculations. Say we have one table with values and one with a mapping like a precalculated log table. Joining both gets the log of each of your values:

select t.value, log.y from table t left outer join log_table log on t.value=log.x

Simple calculations

We can even use a linear interpolation between two values. Say we have only the function values stored for integers but we values between them and these values between them can be interpolated linearly.

select t.value, (t.value-floor(t.value))*f.y + (ceil(t.value)-t.value)*g.y from table t left outer join function_table f on floor(t.value)=f.x left outer join function_table g on ceil(t.value)=g.x

When you need to calculate for large sets of values and insert them into another table it might be better to calculate in SQL and insert in one step without all the conversion and wrapping stuff present in programming languages.

Conditions

Another often overlooked feature is to use a condition:

select case when MOD(t.value, 2) = 0 then 'divisible by 2' else 'not divisible by 2' end from table t

These handful operations are my basic toolbox when working with SQL, almost all queries I need can be formulated with them.

Dates and timestamps

One last reminder: when you work with time always specify the wanted time zone in your query.